Presentation (CTI)

OBJECTIVES: Assessment of comparative effectiveness is key to health technology assessment (HTA) with decision making based on how treatments perform relative to existing standards of care. Methods to compare treatment effectiveness have evolved over time. We sought to describe the approaches used to assess comparative treatment effectiveness and critique of these in HTA by the National Institute for Health and Care Excellence (NICE).
METHODS: Scoping Review methods were followed; the review included final appraisal documents for single and highly specialised technology appraisals published by NICE between 1/1/2025 and 1/1/2026 (n=100). Terminated appraisals were excluded (n=21). Approaches to comparative effectiveness assessment were extracted alongside any evidence review group (ERG) or NICE committees’ critique. Data were extracted by a single reviewer and verified by a second to ensure accuracy. Quantitative and narrative description of findings were given.
RESULTS: Most reported was the use of direct head-to-head trial evidence or network meta-analysis (NMA). Newer or more complex methods aiming to reduce bias when patient characteristics differ across trials are used although not as often. The main criticisms of methods, as highlighted by the ERGs and NICE committees, were regarding study selection, comparability (e.g., heterogeneity in patient cohort characteristics or trial design) and the analysis approach chosen (e.g., choice between indirect treatment comparison (ITC) or NMA), insufficient matching of effect modifiers in matching-adjusted indirect comparisons (MAICs), or insufficient justification of matching approach. The face validity and reproducibility of findings were also criticised.
CONCLUSIONS: Newer meta-analytic methods and adjustment techniques could address some of the common critiques seen and enhance therapeutic value demonstration. The use of conservative methods may reflect limited precedent for the acceptance of findings from complex approaches which introduce additional assumptions and uncertainty alongside concerns regarding clinical expert ability to adequately validate such approaches.