STRUCTURED REVIEW OF AN RWE PROTOCOL TEMPLATE IDENTIFIES UPSTREAM GAPS IN CAUSAL AND FRAMEWORK ALIGNMENT
Author(s)
Gary Schneider, MSPH, ScD.
Delve-RW, Franklin, MA, USA.
Delve-RW, Franklin, MA, USA.
OBJECTIVES: Standardized templates influence how RWE studies are designed, yet their contribution to downstream methodological completeness has not been systematically evaluated. This case study quantifies template-level instructional gaps using a structured, framework-integrated review process.
METHODS: A HARPER-based NIS protocol template was reviewed using the REAL-Q framework, anchored by Quality Assurance Review Items (QARI).QARI was informed by established RWE frameworks and refined through nearly 100 protocol and SAP reviews. The QARI used for this template review contained 132 items across 11 themes. TARGET concepts (Items 1-7[h.ii]) were expanded into 88 operational elements (TARGET-G), and HARPER concept status (explicit/implicit/not reported) was incorporated. QARI was cross-referenced to HARPER and TARGET-G and integrated into REAL-Q for structured review, incorporating unused-QARI reconciliation. Because the unit of analysis was a protocol template rather than a populated protocol, unused QARI were interpreted as opportunities to strengthen template guidance rather than deficiencies in a specific study. Quantitative evaluation therefore focused on HARPER-explicit and TARGET-restricted QARI coverage.
RESULTS: Among 81 in-scope HARPER-explicit QARI, 59 were used (73% QARI coverage). QARI coverage gaps of ≥50% were observed in the study objectives & research questions, bias control, and statistical methods themes. Within TARGET-restricted QARI (n=85), QARI coverage was 66%, with a different gap pattern vs. HARPER-aligned QARI. Notably, HARPER-explicit QARI coverage was 100% for both the study background and sensitivity analyses themes, but decreased to 50% and 32%, respectively, in the TARGET domain, indicating reduced support for causal framing and analytic robustness beyond HARPER-defined concepts.
CONCLUSIONS: Protocol templates serve as methodological infrastructure. Structured assessment demonstrates that instructional gaps in template guidance may propagate downstream into study design and analysis planning, suggesting that template optimization may represent an upstream opportunity to improve RWE quality and transparency for regulatory and HTA decision-making.
METHODS: A HARPER-based NIS protocol template was reviewed using the REAL-Q framework, anchored by Quality Assurance Review Items (QARI).QARI was informed by established RWE frameworks and refined through nearly 100 protocol and SAP reviews. The QARI used for this template review contained 132 items across 11 themes. TARGET concepts (Items 1-7[h.ii]) were expanded into 88 operational elements (TARGET-G), and HARPER concept status (explicit/implicit/not reported) was incorporated. QARI was cross-referenced to HARPER and TARGET-G and integrated into REAL-Q for structured review, incorporating unused-QARI reconciliation. Because the unit of analysis was a protocol template rather than a populated protocol, unused QARI were interpreted as opportunities to strengthen template guidance rather than deficiencies in a specific study. Quantitative evaluation therefore focused on HARPER-explicit and TARGET-restricted QARI coverage.
RESULTS: Among 81 in-scope HARPER-explicit QARI, 59 were used (73% QARI coverage). QARI coverage gaps of ≥50% were observed in the study objectives & research questions, bias control, and statistical methods themes. Within TARGET-restricted QARI (n=85), QARI coverage was 66%, with a different gap pattern vs. HARPER-aligned QARI. Notably, HARPER-explicit QARI coverage was 100% for both the study background and sensitivity analyses themes, but decreased to 50% and 32%, respectively, in the TARGET domain, indicating reduced support for causal framing and analytic robustness beyond HARPER-defined concepts.
CONCLUSIONS: Protocol templates serve as methodological infrastructure. Structured assessment demonstrates that instructional gaps in template guidance may propagate downstream into study design and analysis planning, suggesting that template optimization may represent an upstream opportunity to improve RWE quality and transparency for regulatory and HTA decision-making.
Conference/Value in Health Info
2026-11, ISPOR Europe 2026, Vienna, Austria
Value in Health, Volume 29, Issue 12S
Code
MSR108
Topic
Methodological & Statistical Research, Real World Data & Information Systems, Study Approaches
Topic Subcategory
Confounding, Selection Bias Correction, Causal Inference
Disease
No Additional Disease & Conditions/Specialized Treatment Areas