Tackling Rare Events in Network Meta-Analysis: Comparative Performance of Methods for Sparse Safety Outcomes

OBJECTIVES: Network meta-analysis (NMA) is increasingly employed to compare multiple treatments simultaneously in healthcare research. However, synthesizing safety outcomes often involves sparse networks, with limited direct evidence and/or few studies per comparison. These networks pose significant challenges in robustly estimating effects, particularly for outcomes with zero counts. We aim to identify and compare robust methodological approaches for analysing binary rare event data, to provide recommendations on model choice and implications in regulatory submissions.

METHODS: Simulations were conducted using two sparse network structures: a star-shaped network and one with closed loops. Scenarios varied by the number of treatments and studies informing each comparison, sample size per study arm and the proportion of study arms with zero events. Two frequentist fixed-effect approaches, Mantel-Haenszel NMA and Penalized Likelihood NMA, were evaluated. For Bayesian methods, three models were tested: an exact Binomial model using non-informative priors, a model with non-informative priors and a continuity correction, and a model with more informative priors on the variance and/or treatment effect parameters. Primary estimands were log odds ratios. Performance was evaluated in terms of bias, mean squared error and length of confidence or credible intervals.

RESULTS: In line with previous simulations studies, frequentist and Bayesian methods differed in their performance in scenarios with small sample sizes and low events. As network density increased, differences in performance among the methods became less pronounced. Bayesian methods with non-informative priors often had convergence issues, making them unsuitable for such analyses. Assuming narrower priors mitigated convergence problems and yielded low-bias estimates.

CONCLUSIONS: While recently proposed approaches offer reliable estimates, the choice of approach for sparse NMAs is multi-faceted. The utility of synthesising all-zero event studies need to be justified and sensitivity analyses should always be conducted to ensure robustness of the results.