Integrating Prospective and Retrospective Real-World Data to Construct External Comparator Arms: Novel Methods and Design Considerations
Author(s)
Benjamin Ackerman, PhD1, Craig S. Meyer, MPH, MS, PhD2, Sebastian Schneeweiss, ScD, MD3, Ariel Bourla, MD, PhD1, Daniel Backenroth, PhD1, Syed Safdar, PhD4, Levon Demirdjian, PhD4.
1Johnson & Johnson Innovative Medicine, Raritan, NJ, USA, 2Johnson & Johnson Innovative Medicine, Brisbane, CA, USA, 3Brigham and Women's Hospital and Harvard Medical School, Boston, MA, USA, 4Johnson & Johnson Innovative Medicine, Carlsbad, CA, USA.
1Johnson & Johnson Innovative Medicine, Raritan, NJ, USA, 2Johnson & Johnson Innovative Medicine, Brisbane, CA, USA, 3Brigham and Women's Hospital and Harvard Medical School, Boston, MA, USA, 4Johnson & Johnson Innovative Medicine, Carlsbad, CA, USA.
OBJECTIVES: External comparator arms (ECAs) based on real-world data (RWD) can help contextualize single-arm oncology trials and enable estimation of treatment efficacy for outcomes like overall survival. Prospective RWD studies are ideal for ECAs but may be infeasible due to time and logistics. Retrospective RWD are often more readily available, albeit subject to quality issues like data missingness and unobserved confounding. This study evaluates methods for combining prospective and retrospective RWD to construct ECAs and estimate treatment efficacy, while simultaneously mitigating outcome biases.
METHODS: Simulations were conducted to assess statistical properties when estimating ECA treatment effects (hazard ratios), where comparators combined prospective and retrospective RWD. RWD sample sizes and the degree of unobserved confounding between RWD samples were varied. After aligning RWD populations to the single-arm trial (e.g., via propensity scores), an Empirical Bayes dynamic borrowing (EBDB) method was applied to down-weight retrospective RWD based on dissimilarity of outcome event rates relative to prospective RWD. EBDB performance (bias, type 1 error, coverage) was evaluated against 'naive' pooling of RWD.
RESULTS: When unobserved confounding produces differences in prospective and retrospective outcomes of ~10% (true median survival = 32 months), naively pooling RWD can bias efficacy estimates by ~7-9%, leading to type 1 error of ~10-14% and decreased confidence interval coverage. Applying EBDB to down-weight retrospective RWD reduces bias in treatment effect estimation. Under extreme unobserved confounding (>25%), EBDB essentially discards retrospective RWD with weights close to zero.
CONCLUSIONS: EBDB can safeguard against unobserved confounding between RWD sources after population alignment, increasing power and statistical precision over ECAs using only one RWD source. When designing ECAs, augmenting prospective RWD with retrospective RWD can potentially reduce costs and study duration by requiring less prospectively generated data. Leveraging methods like EBDB can effectively attenuate biases relative to prospective RWD standards, enhancing reliability of outcomes in the pooled ECA.
METHODS: Simulations were conducted to assess statistical properties when estimating ECA treatment effects (hazard ratios), where comparators combined prospective and retrospective RWD. RWD sample sizes and the degree of unobserved confounding between RWD samples were varied. After aligning RWD populations to the single-arm trial (e.g., via propensity scores), an Empirical Bayes dynamic borrowing (EBDB) method was applied to down-weight retrospective RWD based on dissimilarity of outcome event rates relative to prospective RWD. EBDB performance (bias, type 1 error, coverage) was evaluated against 'naive' pooling of RWD.
RESULTS: When unobserved confounding produces differences in prospective and retrospective outcomes of ~10% (true median survival = 32 months), naively pooling RWD can bias efficacy estimates by ~7-9%, leading to type 1 error of ~10-14% and decreased confidence interval coverage. Applying EBDB to down-weight retrospective RWD reduces bias in treatment effect estimation. Under extreme unobserved confounding (>25%), EBDB essentially discards retrospective RWD with weights close to zero.
CONCLUSIONS: EBDB can safeguard against unobserved confounding between RWD sources after population alignment, increasing power and statistical precision over ECAs using only one RWD source. When designing ECAs, augmenting prospective RWD with retrospective RWD can potentially reduce costs and study duration by requiring less prospectively generated data. Leveraging methods like EBDB can effectively attenuate biases relative to prospective RWD standards, enhancing reliability of outcomes in the pooled ECA.
Conference/Value in Health Info
2025-11, ISPOR Europe 2025, Glasgow, Scotland
Value in Health, Volume 28, Issue S2
Code
RWD110
Topic
Methodological & Statistical Research, Real World Data & Information Systems, Study Approaches
Disease
Oncology, Rare & Orphan Diseases