RESIDUAL CONFOUNDING IN REAL-WORLD EXTERNAL CONTROL ARMS: AN UPDATED NICE APPRAISAL REVIEW
Author(s)
Thomas P. Leahy, PhD1, Alex Turner, PhD2, Cormac Sammon, PhD3.
1Arrow Health Economics, Toronto, ON, Canada, 2Arrow Health Economics, Canterbury, United Kingdom, 3Arrow Health Economics, Foxford, Ireland.
1Arrow Health Economics, Toronto, ON, Canada, 2Arrow Health Economics, Canterbury, United Kingdom, 3Arrow Health Economics, Foxford, Ireland.
OBJECTIVES: Real‑world evidence external control arms (RW‑ECAs) are increasingly submitted as comparative effectiveness evidence in NICE technology appraisal (TA) submissions. Residual confounding remains a key issue, particularly in unanchored comparisons. This study extends a previous study to examine how residual confounding was addressed and appraised in recent NICE single TAs (STAs) that included RW-ECAs in the initial submission.
METHODS: A review of NICE STA guidance published in 2025- May 2026 was undertaken. Appraisals that included a RW‑ECA as part of the company’s initial submission were identified. Data were extracted on RWD sources, geography, study design, indirect treatment comparison methodology, alignment strategies between trial and RWD populations, approaches to residual confounding, use of structured sensitivity/bias analyses, and critique by the Evidence Assessment Group (EAG).
RESULTS: The search identified 128 records; following exclusion of terminated submissions, multiple technology appraisals, and STA reviews, 72 STAs remained. Of these, 7 STAs included a RW‑ECA as part of the company’s initial submission. All RW‑ECAs were based on retrospective data. Unanchored methods predominated, including matching‑adjusted indirect comparisons (MAIC), inverse probability weighting (IPW) and regression‑based or doubly robust approaches. Population alignment typically relied on eligibility restriction and re‑weighting on observed prognostic factors. Residual confounding was commonly acknowledged as unavoidable, typically driven by unmeasured or missing key prognostic variables. Structured bias analyses were inconsistently applied; while some submissions or EAGs conducted scenario or sensitivity analyses; negative controls and E-Values were not used in any appraisal. EAGs highlighted residual confounding as a major source of uncertainty, frequently requesting additional analyses, alternative adjustment strategies, or exploration of other control data sources.
CONCLUSIONS: Recent NICE STAs demonstrate growing but heterogeneous use of RW‑ECAs, with residual confounding remaining a central limitation in unanchored comparisons. While acknowledgement of residual bias is universal, the use of approaches to quantify or mitigate it continue to be underutilised.
METHODS: A review of NICE STA guidance published in 2025- May 2026 was undertaken. Appraisals that included a RW‑ECA as part of the company’s initial submission were identified. Data were extracted on RWD sources, geography, study design, indirect treatment comparison methodology, alignment strategies between trial and RWD populations, approaches to residual confounding, use of structured sensitivity/bias analyses, and critique by the Evidence Assessment Group (EAG).
RESULTS: The search identified 128 records; following exclusion of terminated submissions, multiple technology appraisals, and STA reviews, 72 STAs remained. Of these, 7 STAs included a RW‑ECA as part of the company’s initial submission. All RW‑ECAs were based on retrospective data. Unanchored methods predominated, including matching‑adjusted indirect comparisons (MAIC), inverse probability weighting (IPW) and regression‑based or doubly robust approaches. Population alignment typically relied on eligibility restriction and re‑weighting on observed prognostic factors. Residual confounding was commonly acknowledged as unavoidable, typically driven by unmeasured or missing key prognostic variables. Structured bias analyses were inconsistently applied; while some submissions or EAGs conducted scenario or sensitivity analyses; negative controls and E-Values were not used in any appraisal. EAGs highlighted residual confounding as a major source of uncertainty, frequently requesting additional analyses, alternative adjustment strategies, or exploration of other control data sources.
CONCLUSIONS: Recent NICE STAs demonstrate growing but heterogeneous use of RW‑ECAs, with residual confounding remaining a central limitation in unanchored comparisons. While acknowledgement of residual bias is universal, the use of approaches to quantify or mitigate it continue to be underutilised.
Conference/Value in Health Info
2026-11, ISPOR Europe 2026, Vienna, Austria
Value in Health, Volume 29, Issue 12S
Code
HTA213
Topic
Health Technology Assessment, Methodological & Statistical Research, Real World Data & Information Systems
Disease
No Additional Disease & Conditions/Specialized Treatment Areas