Influence of NICE Real-World Evidence Framework on HTA Submissions: Sensitivity Analyses in Real-World External Control Arm Studies
Author(s)
Thomas P. Leahy, PhD1, Craig Keenan, MSc2, Emily Robertshaw, MSc3, Sylvaine Barbier, MSc4, Megan McStravick, MSc3, Alex James Turner, BSc, MSc, PhD3.
1Putnam, TORONTO, ON, Canada, 2Putnam, London, United Kingdom, 3Putnam, Newcastle Upon Tyne, United Kingdom, 4Putnam, LYON, France.
1Putnam, TORONTO, ON, Canada, 2Putnam, London, United Kingdom, 3Putnam, Newcastle Upon Tyne, United Kingdom, 4Putnam, LYON, France.
OBJECTIVES: Growing reliance on single-arm trials (SATs) is driving increased real-world external control arm (RW-ECA) use in health technology assessment (HTA) submissions. However, concerns over residual confounding and unmeasured bias often limit acceptability. The National Institute for Health and Care Excellence (NICE) real-world evidence framework encourages the use of sensitivity analyses, including quantitative bias analysis (QBA), to explore this uncertainty. This study evaluates whether publication of the framework has corresponded with greater adoption of QBA methods and sensitivity analyses in NICE submissions using RW-ECAs.
METHODS: A targeted review of NICE single technology appraisals (STAs) was conducted to identify submissions incorporating RW-ECAs. Data on the use of sensitivity analyses, including QBA, were extracted and compared across two periods, (1) prior to likely adoption of framework and (2) likely post-framework adoption. Additional insights on bias and uncertainty were drawn from Evidence Review Group (ERG) reports.
RESULTS: Of 335 submissions reviewed, 25 included a RW-ECA alongside an SAT, most commonly in oncology (22/25). Only 2 of the 25 submissions implemented formal QBA methods, while 19 conducted broader sensitivity or scenario analyses, for example, sensitivity to different model specifications or data curation decisions. Where QBA was used, it was typically limited in scope and lacked structured methodology. In most cases, companies acknowledged residual confounding without formally quantifying its impact. ERGs frequently highlighted this as a source of residual uncertainty (14/25). Use of RW-ECA in submissions increased from 6.7% (2019-2023) to 10.9% in 2024, however there was no increase in QBA use compared to previous years.
CONCLUSIONS: Despite NICE’s methodological recommendations, QBA remains largely absent from RW-ECA submissions based on one year of likely post-framework adoption. There is a clear opportunity for companies to enhance the robustness of their submissions by incorporating QBA or sensitivity analyses to address residual bias.
METHODS: A targeted review of NICE single technology appraisals (STAs) was conducted to identify submissions incorporating RW-ECAs. Data on the use of sensitivity analyses, including QBA, were extracted and compared across two periods, (1) prior to likely adoption of framework and (2) likely post-framework adoption. Additional insights on bias and uncertainty were drawn from Evidence Review Group (ERG) reports.
RESULTS: Of 335 submissions reviewed, 25 included a RW-ECA alongside an SAT, most commonly in oncology (22/25). Only 2 of the 25 submissions implemented formal QBA methods, while 19 conducted broader sensitivity or scenario analyses, for example, sensitivity to different model specifications or data curation decisions. Where QBA was used, it was typically limited in scope and lacked structured methodology. In most cases, companies acknowledged residual confounding without formally quantifying its impact. ERGs frequently highlighted this as a source of residual uncertainty (14/25). Use of RW-ECA in submissions increased from 6.7% (2019-2023) to 10.9% in 2024, however there was no increase in QBA use compared to previous years.
CONCLUSIONS: Despite NICE’s methodological recommendations, QBA remains largely absent from RW-ECA submissions based on one year of likely post-framework adoption. There is a clear opportunity for companies to enhance the robustness of their submissions by incorporating QBA or sensitivity analyses to address residual bias.
Conference/Value in Health Info
2025-11, ISPOR Europe 2025, Glasgow, Scotland
Value in Health, Volume 28, Issue S2
Code
HTA201
Topic
Health Technology Assessment, Methodological & Statistical Research
Disease
No Additional Disease & Conditions/Specialized Treatment Areas