OBJECTIVES: Various unanchored indirect comparison methods are emerging give the absence of head-to-head RCTs. This study aims to compare the performance between matching-adjusted indirect comparison (MAIC) and naïve treatment indirect comparison (NIC) on survival outcomes.

METHODS: A simulation study was conducted based on a large number of simulated trial data sets generated by Monte Carlo approach. A total of 729 (3⁶) simulated scenarios were created by performing a full factorial arrangement of six conditional factors with three levels for each factor, including the sample size of individual patient data, the sample size of aggregate data, the strength of association between covariates and outcomes, the strength of covariates correlation, the overlap of covariates and the strength of relative treatment effect. MAIC and NIC were then used to estimate the hazard ratio based on the Cox proportional hazards models. Four indexes including the bias, confidence interval coverage, mean square error (MSE) and empirical standard error (ESE) were used to quantify the performance of unbiasedness, randomization validity, precision and efficiency between MAIC and NIC respectively.

RESULTS: When the treatment effect was set to zero, MAIC yielded less biased estimates than NIC in all (243/243) scenarios. When the treatment effect was non-zero, MAIC exhibited systematic bias since the hazard ratio is non-collapsible, even leading to greater bias than NIC in 8% (39/486) scenarios. In terms of randomization validity, MAIC provided higher confidence interval coverage than NIC in 83% (604/729) scenarios. The MSE and ESE of MAIC were larger than NIC in 31% (224/729) and 76% (552/729) scenarios, respectively, indicating that MAIC exhibited less precision and efficiency in some specific scenarios.

CONCLUSIONS: MAIC generates more accurate estimates than NIC, especially under the condition of poor overlap of covariates with the increase of sample size and strength of covariates, despite the increased uncertainty. Caution is needed when interpreting MAIC results.