OBJECTIVES: Population-adjusted ITC, simulated treatment comparison (STC) and matching-adjusted indirect comparison (MAIC), enable comparative effectiveness of therapies for use in health technology assessments in absence of head-to-head trials. While these methods adjust for baseline characteristics differences between-study, the degree of imbalance may impact the outcomes regression model. This simulation evaluated the magnitude of covariate imbalance at different effect modification strengths on population adjusted estimates of treatment efficacy.

METHODS: Patient-level data were simulated using ‘wakefield’ package in R. We set the baseline characteristics (sample size, mean age) and the parameters of association. The comparator trial was varied by increasing the imbalance of age and effect modification strength by increments of 5 years (up to 30) and -0.025 log-odds (up to -0.100), respectively, for 24 permutations total, 1000 iterations each. Bias was estimated as the log-odds of the treatment difference calculated using STC or MAIC minus the true treatment difference, and coverage as whether the simulated 95% confidence intervals contained the true value. Results were summarized using linear regression and means.

RESULTS: The adjustment models demonstrated diminished bias relative to naïve comparison as effect modification strengthened and age imbalance increased. There were no linear trends in increased bias observed from increasing between-study age imbalance (STC, p=0.731; MAIC, p=0.809) or effect modification strength (STC, p=0.124; MAIC, p=0.293). The mean bias from the true treatment value of -0.4 when the covariate imbalance was 5 years was 0.02 and 0.00, and at 30 years, 0.02 and 0.00, STC and MAIC respectively. Bias was similarly absent at varying strengths of effect modification. The coverage summary estimated the confidence intervals contained the true treatment difference throughout all simulation permutations at least 94% of the time.

CONCLUSIONS: STC and MAIC perform without bias within plausible ranges of effect modification strength when there are strongly imbalanced covariates between studies.