OBJECTIVES : Population-adjusted indirect comparisons such as matching-adjusted indirect comparisons (MAICs) and simulated treatment comparisons (STCs) are becoming increasingly common in health technology assessments when conventional treatment comparison techniques are not feasible. Despite current National Institute for Health and Care Excellence (NICE) guidelines not displaying a preference for conducting a MAIC over an STC, MAICs are more commonly used. This study evaluated recent NICE single technology appraisals (STAs) that used STCs to determine how their use is perceived by the Evidence Review Group (ERG) and appraisal committee.

METHODS : The NICE website was searched for the keywords “simulated treatment comparison” and “STC”. All documents unrelated to STAs and any published before the release of NICE Decision Support Unit Technical Support Document 18 (December 2016) were excluded. Details of the submission, rationale for performing an STC, methodology, criticisms and response were assessed and summarised. All data extraction was performed by two independent reviewers.

RESULTS : Four NICE STAs were identified that used STC methodology. All STAs were oncology submissions where STCs were used to simulate survival outcomes such as progression-free and overall survival. The main rationale for conducting STCs were lack of comparative efficacy data and presence of single arm trial data. Two STAs provided reasoning for performing an STC over a MAIC and two STAs also conducted a MAIC. Mixed responses were given by the ERG, with main negative feedback being lack of validity and omission of prognostic variables and effect modifiers. The appraisal committee mainly agreed with the responses provided by the ERG

CONCLUSIONS : Despite current guidance, STCs are rarely used in NICE STAs and are generally not well received by the ERG and appraisal committee. Many criticisms were not unique to STC methodology and therefore further research into why MAICs are more commonly implemented than STCs is warranted.