TOOLS FOR ASSESSING RISK OF BIAS IN REAL-WORLD EVIDENCE: A SCOPING REVIEW
Author(s)
Wendy Nieto Gutiérrez, MSc1, Silvia Moler, PhD1, Sebastián Medina-Ramírez, MSc2, Soledad Isern de Val, PhD1.
1Aragon Health Sciences Institute, Zaragoza, Spain, 2Unidad de Investigación, Instituto Nacional de Ciencias Neurológicas, Lima, Peru.
1Aragon Health Sciences Institute, Zaragoza, Spain, 2Unidad de Investigación, Instituto Nacional de Ciencias Neurológicas, Lima, Peru.
OBJECTIVES: Real-world evidence (RWE) is increasingly used in regulatory and healthcare decision-making, yet many studies face methodological limitations that may introduce bias. Existing reviews have tended to focus on generic observational study tools rather than instruments specifically designed for RWE. This study aimed to map and characterise tools developed, adapted or validated to assess risk of bias (RoB) in RWE studies.
METHODS: We conducted a scoping review of studies and documents describing RoB tools for RWE. PubMed and Embase were searched, complemented by targeted Google searches of health technology assessment agencies and decision-maker websites. Tool characteristics, development procedures, judgement approaches, and type of data science task (description, prediction and counterfactual prediction) were extracted. Items were mapped to stages of RWE generation and four bias domains: selection, confounding, measurement and reporting bias.
RESULTS: Eleven tools published between 2016 and 2025 were included. Most targeted primary longitudinal RWE studies and causal or comparative questions. Most used qualitative or categorical judgements; QATSM-RWS was the only tool with explicit numerical scoring. Reporting of development procedures was inconsistent. Mapping showed substantial heterogeneity: some tools emphasised study design, whereas others focused on data quality or data analysis, with interpretation less consistently addressed. Selection and confounding bias were most frequently represented, while measurement and reporting bias were covered more variably.
CONCLUSIONS: RoB tools for RWE are heterogeneous and should not be treated as interchangeable. Tool selection should reflect the study objective, data source, design and expected sources of bias; complementary tools may be required when important stages or domains are insufficiently covered.
METHODS: We conducted a scoping review of studies and documents describing RoB tools for RWE. PubMed and Embase were searched, complemented by targeted Google searches of health technology assessment agencies and decision-maker websites. Tool characteristics, development procedures, judgement approaches, and type of data science task (description, prediction and counterfactual prediction) were extracted. Items were mapped to stages of RWE generation and four bias domains: selection, confounding, measurement and reporting bias.
RESULTS: Eleven tools published between 2016 and 2025 were included. Most targeted primary longitudinal RWE studies and causal or comparative questions. Most used qualitative or categorical judgements; QATSM-RWS was the only tool with explicit numerical scoring. Reporting of development procedures was inconsistent. Mapping showed substantial heterogeneity: some tools emphasised study design, whereas others focused on data quality or data analysis, with interpretation less consistently addressed. Selection and confounding bias were most frequently represented, while measurement and reporting bias were covered more variably.
CONCLUSIONS: RoB tools for RWE are heterogeneous and should not be treated as interchangeable. Tool selection should reflect the study objective, data source, design and expected sources of bias; complementary tools may be required when important stages or domains are insufficiently covered.
Conference/Value in Health Info
2026-11, ISPOR Europe 2026, Vienna, Austria
Value in Health, Volume 29, Issue 12S
Code
MSR178
Topic
Health Technology Assessment, Methodological & Statistical Research, Real World Data & Information Systems
Topic Subcategory
Confounding, Selection Bias Correction, Causal Inference
Disease
No Additional Disease & Conditions/Specialized Treatment Areas