OBJECTIVES: Numerous software are available for performing Bayesian network meta-analysis (NMA), however WinBUGS remains the preferred option, with code provided in the National Institute of Health and Care Excellence Technical Support Document widely adopted. Alternative software may offer varying performance, running time and user experience. This case study compares results using different software to perform NMAs of binomial outcomes assessing the relative effects of treatments for patients with ulcerative colitis.

METHODS: Previously published NMAs conducted using OpenBUGS were replicated using the R packages R2WinBUGS and multinma to run WinBUGS and Stan, respectively, and their results were compared. Analyses were stratified by prior exposure to anti-tumour necrosis factor (anti-TNF) therapy and, data allowing, comprised the following outcomes at induction and maintenance: response, remission, mucosal healing, and discontinuations due to adverse events. Results from 13 NMAs were compared in terms of point estimates (odds ratios [ORs]), credible intervals (CrI), treatment rankings, and subsequent conclusions, as well as considering user experience.

RESULTS: Outputs were similar across software for all NMAs. For example, mean ORs for infliximab versus adalimumab were consistent across the different software in the fixed-effect (FE) NMA of response in anti-TNF naïve patients at induction: 2.19 [95% CrI: 1.35-3.55] using OpenBUGS vs. 2.51 [1.52-3.95] using WinBUGS vs. 2.44 [1.51-3.93] using Stan. Further, conclusions aligned across software, with no changes to treatment rankings. All networks were too small to give reliable estimates for random-effects models using OpenBUGS and WinBUGS, but comparable results to FE analyses were observed using Stan (2.43 [1.34-4.38]). Finally, Stan required less running time and was less computationally demanding.

CONCLUSIONS: Comparing software resulted in consistent outcomes and interpretations. While these results may not be generalizable to all network shapes and sizes, this study demonstrates that Stan provides a valid alternative and may be advantageous both in terms of performance and user experience.