OBJECTIVES: Matching-adjusted indirect comparison (MAIC) is the most common methodology considered in technology appraisals for pairwise comparisons that control for imbalances in baseline characteristics. The aim of this simulation study was to assess the performance of different methods for estimating the uncertainty around treatment effects derived via anchored MAIC.

METHODS: Monte Carlo simulations (n=1,000 replications) were conducted for a total of 18 scenarios to investigate the impact of outcome type (binary, survival), sample size, and population overlap on variance estimation in MAICs. In each scenario, weighted logistic regression and Cox proportional hazards models were fitted for binary and survival outcomes using four different methods for variance estimation: 1) conventional estimators (CE) using raw weights; 2) CE using weights rescaled to the effective sample size (ESS); 3) robust sandwich estimators; and 4) bootstrapping. The performance of each method was evaluated on the basis of empirical coverage of 95% confidence intervals (CI) and the ratio of average estimated standard error (SE) versus empirical SE.

RESULTS: The empirical coverage for CE + ESS-scaled weights (ranging from 94.5% to 96.4%) did not significantly deviate from the nominal confidence level in 17 of 18 scenarios. On the contrary, variance was underestimated by CE + raw weights (6 of 6 scenarios), bootstrapping (4/6), and sandwich estimators (3/6) in the scenarios with poor population overlap (~77% reduction in the ESS). The use of CE + raw weights underestimated variance in half of the scenarios with moderate overlap, while all other methods had unbiased results. All four methods provided accurate estimates of the variance in scenarios with strong overlap, with coverage and SE ratios between 94.2%-96.2% and 0.97-1.06.

CONCLUSIONS: The extent of population overlap is an important consideration for variance estimation in MAICs. The use of CE + ESS-scaled weights produced SEs and CIs that were fairly precise across all scenarios.