OBJECTIVES : To assess the performance of population adjustment methods for indirect comparisons, in a range of scenarios and under various assumptions, through a simulation study.

METHODS : Standard anchored indirect comparisons use aggregate data and assume that any effect modifiers (EMs) are balanced between populations. Population adjustment methods aim to relax this assumption using individual patient data (IPD) from one or more studies to adjust for differences in EMs between populations. Current methods include Matching-Adjusted Indirect Comparison (MAIC), Simulated Treatment Comparison (STC), and Multilevel Network Meta-Regression (ML-NMR).

All population adjustment methods assume that there are no missing EMs, and typically share EMs for active treatments. ML-NMR and STC may also extrapolate outside the IPD population, but MAIC cannot. We undertook a simulation study to examine the performance of each method (including standard indirect comparisons) and the impact of breaking these assumptions. We assessed bias, standard error, and coverage.

RESULTS : ML-NMR and STC performed similarly throughout, eliminating bias and providing accurate estimates of standard errors when the assumptions were met, but remaining biased when they were not. MAIC performed poorly in almost all scenarios, in some cases increasing bias compared with a standard indirect comparison. MAIC required full overlap between populations, otherwise estimates were biased and standard errors unstable, especially when sample size was small. All methods incurred bias when EMs were missing from the model.

CONCLUSIONS : Serious questions are raised about the suitability of MAIC, which is only valid in scenarios where there is likely to be little benefit over a standard indirect comparison. ML-NMR and STC are both robust methods for population adjustment, but careful and considered selection of potential EMs prior to analysis is necessary to avoid bias. ML-NMR offers additional advantages, including synthesising larger treatment networks, relaxing the shared EM assumption, and producing estimates in any target population.