Combining Causal Inference and Within-Trial Economic Evaluation Methods to Assess the Cost-Effectiveness of a Mental Health Service Using Real-World Data: The Quasi-Experimental Adapt Study


Franklin M1, Porter A2, De Vocht F2, Kearns B3, Latimer N4, Hernandez M5, Young T5, Kidger J2
1University of Sheffield, Sheffield, NYK, UK, 2University of Bristol, Bristol, South West, UK, 3Lumanity, Sheffield, UK, 4University of Sheffield & Delta Hat Limited, Sheffield, DBY, Great Britain, 5University of Sheffield, Sheffield, South Yorkshire, UK

OBJECTIVES: Real-world evidence is playing an increasingly important role in health technology assessment, but is prone to selection and confounding bias. We demonstrate how to conduct a real-world within-study cost-per-quality-adjusted-life-year (QALY) analysis. We combined traditional within-trial bootstrapped regression-baseline-adjustment with causal inference methods, using a Target Trial (TT) framework, inverse probability weights (IPWs), marginal structural models (MSMs), and g-computation, applied to England’s Improving Access to Psychological Therapies (IAPT) mental-health e-records.

METHODS: The ‘Assessing a Distinct IAPT service’ (ADAPT) quasi-experimental-study evaluated an Enhanced-IAPT service Vs. IAPT’s treatment-as-usual. IAPT collects patient-reported PHQ-9-depression and GAD-7-anxiety scores at index-assessment and each treatment session, from which we predicted EQ-5D utilities using a mapping function. Our primary estimands were incremental costs and QALYs for Enhanced Vs. treatment-as-usual at 16-weeks post-IAPT-index-assessment.

We prespecified our TT including eligibility, treatment strategies, assignment procedure, follow-up, outcomes, estimands, and analysis plan. We used stabilised treatment-related and censoring-related IPWs within MSMs to reduce selection and confounding bias due to non-randomised treatment allocation and informative censoring, respectively. Our doubly-robust approach involved MSM-adjusted baseline covariates and g-computation to estimate incremental utilities, costs, and QALYs, with bootstrapped bias-corrected 95% confidence-intervals (95%bCIs) and cost-effectiveness acceptability curves.

RESULTS: Analysis sample: Enhanced, N=5,441; treatment-as-usual, N=2,149. Naïve regression-baseline-adjustment and doubly-robust approaches suggested Enhanced-IAPT dominated treatment-as-usual, with average per-person (95%bCIs) cost-savings of £30.64 (£22.26 to £38.90) or £29.64 (£20.69 to £37.99) and QALYs-gained of 0.00035 (-0.00075 to 0.00152) or 0.00052 (-0.00105 to 0.00277), respectively; probability of cost-effectiveness at £30,000 per QALY was 99% or 95%, respectively. The doubly-robust and naïve results concurred; albeit, the doubly-robust results suggested average QALY gains were higher but less certain. The cost-effectiveness results were driven by the potential for the Enhanced service to provide cost-savings.

CONCLUSIONS: When using real-world data and treatment allocation is non-randomized, the TT framework alongside doubly-robust analyses should be used to reduce selection and confounding bias.

Conference/Value in Health Info

2023-11, ISPOR Europe 2023, Copenhagen, Denmark

Value in Health, Volume 26, Issue 11, S2 (December 2023)




Economic Evaluation, Methodological & Statistical Research, Study Approaches

Topic Subcategory

Confounding, Selection Bias Correction, Causal Inference, Cost-comparison, Effectiveness, Utility, Benefit Analysis, Electronic Medical & Health Records, Trial-Based Economic Evaluation


Mental Health (including addition), No Additional Disease & Conditions/Specialized Treatment Areas

Your browser is out-of-date

ISPOR recommends that you update your browser for more security, speed and the best experience on Update my browser now