Indirect Treatment Comparisons in Health Technology Assessment Submissions: A Review and Critique of Best Practice
Author(s)
Bodille Blomaard, MSc1, Claire Ainsworth, BSc, MSc2, Lytske Bakker, PhD3.
1OPEN Health Group, Rotterdam, Netherlands, 2OPEN Health Group, Manchester, United Kingdom, 3OPEN Health, Berkel en Rodenrijs, Netherlands.
1OPEN Health Group, Rotterdam, Netherlands, 2OPEN Health Group, Manchester, United Kingdom, 3OPEN Health, Berkel en Rodenrijs, Netherlands.
OBJECTIVES: In the absence of direct evidence, indirect treatment comparisons (ITCs) are often used to assess the comparative effectiveness of novel treatments. Despite established best-practice guidelines, insight into the adherence of ITCs to these guidelines is lacking. This study reviewed the methods used in technology appraisals (TAs) submitted to the National Institute for Health and Care Excellence (NICE) along with adherence to related guidelines.
METHODS: A targeted review of the NICE website was performed to identify TAs published between April 2022-March 2025 reporting matching-adjusted indirect comparisons (MAICs), network meta-analyses (NMAs), simulated treatment comparisons (STCs), or multilevel network meta-regression (ML-NMR). Data on ITC methodology used and relevant critique of submissions was extracted.
RESULTS: Of 257 TAs identified, 114 met the inclusion criteria. NMAs and MAICs were most frequently used (61.4%, 48.2%) whereas STCs (7.9%) and ML-NMRs (1.8%) were solely included as sensitivity analyses when multiple methods were used (32.4%). Common concerns of evidence review groups (ERGs) were heterogeneity in patient characteristics in NMAs (79%), missing treatment effect modifiers and prognostic variables in MAICs (76%) and misalignment between the evidence presented and the target population of interest (NMA:24%, MAIC:44%). In 23% of NMAs, companies favoured fixed-effects models while the ERG preferred random-effects (RE). However, this percentage varied considerably across the three years (2022:39%, 2023:27%, 2024:8%). Furthermore, the use of informative priors increased in RE models for NMAs (2022:6%, 2023:4%, 2024:46%) and ERG requests to add them declined (2022:50%, 2023:12%, 2024:12%).
CONCLUSIONS: Enabling the use of more complex methods through publications (e.g., on informative priors by Turner et al.), and software (e.g. multinma), may increase guideline adherence in TAs. Persistent challenges in MAICs and NMAs, such as bias from treatment effect modification and population misalignment remain. Here, the adoption of ML-NMR can offer a more robust alternative to extensive sensitivity analyses and enhancing the credibility of results.
METHODS: A targeted review of the NICE website was performed to identify TAs published between April 2022-March 2025 reporting matching-adjusted indirect comparisons (MAICs), network meta-analyses (NMAs), simulated treatment comparisons (STCs), or multilevel network meta-regression (ML-NMR). Data on ITC methodology used and relevant critique of submissions was extracted.
RESULTS: Of 257 TAs identified, 114 met the inclusion criteria. NMAs and MAICs were most frequently used (61.4%, 48.2%) whereas STCs (7.9%) and ML-NMRs (1.8%) were solely included as sensitivity analyses when multiple methods were used (32.4%). Common concerns of evidence review groups (ERGs) were heterogeneity in patient characteristics in NMAs (79%), missing treatment effect modifiers and prognostic variables in MAICs (76%) and misalignment between the evidence presented and the target population of interest (NMA:24%, MAIC:44%). In 23% of NMAs, companies favoured fixed-effects models while the ERG preferred random-effects (RE). However, this percentage varied considerably across the three years (2022:39%, 2023:27%, 2024:8%). Furthermore, the use of informative priors increased in RE models for NMAs (2022:6%, 2023:4%, 2024:46%) and ERG requests to add them declined (2022:50%, 2023:12%, 2024:12%).
CONCLUSIONS: Enabling the use of more complex methods through publications (e.g., on informative priors by Turner et al.), and software (e.g. multinma), may increase guideline adherence in TAs. Persistent challenges in MAICs and NMAs, such as bias from treatment effect modification and population misalignment remain. Here, the adoption of ML-NMR can offer a more robust alternative to extensive sensitivity analyses and enhancing the credibility of results.
Conference/Value in Health Info
2025-11, ISPOR Europe 2025, Glasgow, Scotland
Value in Health, Volume 28, Issue S2
Code
CO151
Topic
Clinical Outcomes, Health Policy & Regulatory
Topic Subcategory
Comparative Effectiveness or Efficacy
Disease
No Additional Disease & Conditions/Specialized Treatment Areas