VALIDATION OF THE REAL-Q (REAL-WORLD EVALUATION ASSESSMENT LIST FOR QUALITY) TOOL
Author(s)
Gary Schneider, MSPH, ScD1, Betsy J. Lahue, MPH2;
1Delve-RW, Founder, Franklin, MA, USA, 2Alkemi, Manchester Center, VT, USA
1Delve-RW, Founder, Franklin, MA, USA, 2Alkemi, Manchester Center, VT, USA
OBJECTIVES: Growing acceptance of real-world evidence (RWE) by HTA and regulatory agencies has expanded observational research guidance. For causal effects, HARPER and TARGET provide protocol and target trial emulation (TTE) frameworks, but neither ensures fully transparent RWE statistical analysis plans (SAPs).
METHODS: We developed REAL-Q, an RWE SAP review platform anchored by Quality Assurance Review Items (QARI). QARI was informed by existing frameworks and refined through >75 SAP reviews; as of 2025Q4 it comprised 132 items across 11 themes. TARGET concepts (Items 1-7[h.ii]) were expanded into 65 SAP-level elements (TARGET-G), and HARPER concept status (explicit/implicit/not reported) was incorporated. QARI was cross-referenced to HARPER and TARGET-G and integrated into REAL-Q for structured review and quantitative assessment. The updated QARI was applied to two previously reviewed SAPs, one using a TTE framework. QARI were classified at initial review as adequately addressed or requiring clarification based on reviewer judgment documented in structured comments.
RESULTS: Across two SAPs, 47-55% of QARI were judged adequately addressed at initial review. Patterns (% addressed) differed across SAPs consistent with TTE use. In the non-TTE SAP, coverage was strongly associated with HARPER status (83% vs 50% vs 6% for explicit/implicit/not reported). In the TTE SAP, 57-63% of explicit or implicit concepts and 39% of not-reported concepts were addressed. TARGET-G revealed additional gaps: in the non-TTE SAP, 87% of TARGET-mapped items were addressed vs 18% of non-TARGET-mapped items; the TTE SAP showed a smaller but persistent gap (66% vs 47%).
CONCLUSIONS: Integrating RWE frameworks into structured SAP review enables quantitative assessment of methodological completeness and transparency for regulatory and access decision-making. Differences between TTE and non-TTE SAPs suggest TARGET structure may support more complete and transparent causal analysis planning.
METHODS: We developed REAL-Q, an RWE SAP review platform anchored by Quality Assurance Review Items (QARI). QARI was informed by existing frameworks and refined through >75 SAP reviews; as of 2025Q4 it comprised 132 items across 11 themes. TARGET concepts (Items 1-7[h.ii]) were expanded into 65 SAP-level elements (TARGET-G), and HARPER concept status (explicit/implicit/not reported) was incorporated. QARI was cross-referenced to HARPER and TARGET-G and integrated into REAL-Q for structured review and quantitative assessment. The updated QARI was applied to two previously reviewed SAPs, one using a TTE framework. QARI were classified at initial review as adequately addressed or requiring clarification based on reviewer judgment documented in structured comments.
RESULTS: Across two SAPs, 47-55% of QARI were judged adequately addressed at initial review. Patterns (% addressed) differed across SAPs consistent with TTE use. In the non-TTE SAP, coverage was strongly associated with HARPER status (83% vs 50% vs 6% for explicit/implicit/not reported). In the TTE SAP, 57-63% of explicit or implicit concepts and 39% of not-reported concepts were addressed. TARGET-G revealed additional gaps: in the non-TTE SAP, 87% of TARGET-mapped items were addressed vs 18% of non-TARGET-mapped items; the TTE SAP showed a smaller but persistent gap (66% vs 47%).
CONCLUSIONS: Integrating RWE frameworks into structured SAP review enables quantitative assessment of methodological completeness and transparency for regulatory and access decision-making. Differences between TTE and non-TTE SAPs suggest TARGET structure may support more complete and transparent causal analysis planning.
Conference/Value in Health Info
2026-05, ISPOR 2026, Philadelphia, PA, USA
Value in Health, Volume 29, Issue S6
Code
MSR183
Topic
Methodological & Statistical Research
Topic Subcategory
Confounding, Selection Bias Correction, Causal Inference
Disease
No Additional Disease & Conditions/Specialized Treatment Areas