Assessing the Quality of Real-World Data and Real-World Evidence in Oncology Research: A Cohesive Framework for Researchers

OBJECTIVES: Based on a literature review of recent publications that assessed RWD quality, we propose a new tool for assessing the quality of RWD and Real-World Evidence (RWE) for oncology studies.

METHODS: Published methods for determining the quality of RWD and RWE were identified through a literature review. Results were summarized and unmet needs in oncology research were discussed with subject-matter experts. We developed a new method that focuses on understanding whether (and what) oncology research questions can be answered with the RWD available. Our tool organizes RWD into 9 domains (which were used as our operational definition of quality): (1) conformance, (2) completeness, (3) consistency, (4) accuracy, (5) scalability, (6) timeliness, (7) generalizability, (8) validity, and (9) transparency. Within each domain, 3 categories of quality were defined: high, moderate, and low. The final RWD quality rating is: high quality if all of the first 6 domains had high quality, moderate quality if at least 4 domains had high quality and no domains had low quality, and low quality otherwise. The RWE is rated as follows: high quality if all of the 9 domains had high quality, moderate quality if at least 7 domains had high quality and no domains had low quality, and low quality otherwise.

RESULTS: Literature review identified multiple frameworks and numerous good practices. We developed this simple tool for evaluating the quality of RWD and RWE tailored to oncology, and are in the process of refining and validating this tool. More detailed methods and results from applying this tool to RWD from US Oncology Network will be presented.

CONCLUSIONS: Existing methods are available for identifying fit-for-purpose RWD. A new tool has been developed for oncology research. This has important implications for generating RWE to understand the natural history of disease and the effectiveness of medical products.