Presentation (CTI)

OBJECTIVES: Under the EU JCA, multiplicity becomes more visible as one submission must address multiple PICOs, outcomes, subgroups, and sensitivity analyses. In matching-adjusted indirect comparisons (MAICs), related analyses may rely on shared weighting structures, increasing the risk that apparently favourable effects arise by chance. We quantified the impact of conducting multiple related MAIC analyses on the family-wise error rate (FWER) and assessed whether standard multiplicity corrections control this risk.
METHODS: We simulated unanchored MAICs varying sample size, covariate overlap, number of outcomes, outcome correlation, and number of weighting scenarios. After confirming each individual test maintained the nominal 5% type I error rate, we estimated FWER and power under three treatment-effect patterns: no true effect on any outcome, true effects on all outcomes, and true effects on some outcomes only. Correction procedures were applied to assess the trade-off between FWER control and power loss.
RESULTS: FWER, defined as the probability of at least one false-positive finding across the family of tests, increased as the number of outcomes and weighting scenarios increased, exceeding the nominal level. Greater outcome correlation and covariate overlap consistently attenuated FWER inflation across simulated scenarios. Multiplicity corrections reduced FWER but further decreased power, particularly in low sample size settings.
CONCLUSIONS: In MAICs, multiplicity can make nominally significant findings difficult to interpret in isolation, since the chance of a spurious finding rises with each additional outcome and weighting scenario. Correction procedures can reduce false-positive findings, but may be of limited practical value when many analyses are decision-relevant and sample sizes are low. The JCA setting calls for a pragmatic multiplicity strategy that combines pre-specification of the primary estimand, of clinically meaningful difference thresholds, distinction between confirmatory and supportive analyses, transparent reporting of all comparisons, and interpretation of p-values as indicators of evidence strength rather than definitive hypothesis tests.