OBJECTIVES: Matching adjusted indirect comparisons (MAIC) are used to facilitate population-adjusted indirect treatment comparisons (ITCs) to support health technology assessment submissions. A recent publication (Phillippo 2020, Statistics in Medicine) evaluated the performance of MAIC and found that MAIC may be biased in cases of small sample sizes (N<500), large reductions in effective sample size (ESS), and missing effect modifiers. Therefore, we sought to assess MAICs that have been submitted to the Canadian Agency for Drugs and Technologies in Health (CADTH) to identify and summarize features that may impact the validity of the ITC.

METHODS: CADTH submissions in the pan-Canadian Oncology Drug Review reimbursement review database were queried from 2016 to 2021 to identify submissions that included MAIC. Factors that may impact the validity of the MAIC were extracted including sample sizes, reduction in ESS, and missing effect modifiers. Additional submission details such as final determination of clinical benefit and recommendation were also extracted. Factors were summarized using frequencies and proportions for categorical variables and median/range for continuous variables.

RESULTS: The review identified 22 submissions including an MAIC, of which 20 reported detailed methodological summaries. Median sample sizes were 124.5 (range: 21-470) in the index trial and 138 (range: 14-444) in the comparator trial. ESS was reported in 11/20 trials (55%), with a median of 40.4% reduction (range: 5.1%-75.6%). The ITCs included a median of 9 (range: 3-14) effect modifiers, and 7/20 -submissions reported missing effect modifiers. Furthermore, CADTH noted limitations with respect to the representativeness of the target population in 4/22 (18.2%) submissions. Ultimately, 16/22 (72.7%) submissions had a positive clinical benefit determination resulting in a reimbursement recommendation.

CONCLUSIONS: Several features which may impact the validity of MAICs were present in submissions to CADTH. Given the prevalence of these methodological issues, the use of MAICs as the population-adjustment technique of choice should be discouraged.