OBJECTIVES:

Ample software tools are available to conduct a network meta-analysis (NMA), with each offering varying degrees of functionality and ease of use. A case study compared two software packages (WinBUGS and Stan) for Bayesian NMA of binary outcomes.

METHODS:

The NMA used publicly available data on comparative safety of different treatment regimens for patients living with human immunodeficiency virus (HIV). The network included 9 trials, 8 treatments and contained direct and indirect evidence. The Bayesian NMA with a binomial likelihood was performed in both WinBUGS and Stan and allowed for a comparison of numerical results (odds ratios [OR], 95% credible intervals [CrI]), treatment rankings, residual deviance, computation time and ease of use. Stan was implemented in R using the multinma package and WinBUGS in R using the R2WinBUGS package.

RESULTS:

Differences in the median OR ranged from –0.065 to 0.003. The CrIs from Stan were narrower, with differences in the lower bounds between -0.015 and 0.003 and the upper bounds between -0.572 and 0.008 compared with those from WinBUGS. Treatment rankings and the interpretation of the results did not change. The posterior distributions from WinBUGS displayed right-skewness in comparison to those from Stan. The OR estimates from Stan were generally smaller, potentially due to the difference in the sampling algorithms from WinBUGS. Stan (run through multinma) improved computation time and provided more informative console messages than WinBUGS.

CONCLUSIONS:

Using Stan and WinBUGS for a binomial NMA resulted in comparable outcomes and interpretations in this case study. Stan provided advantages regarding computation time. User experience was also improved compared to WinBUGS. Whilst model results may not be generalizable to all comparisons between WinBUGS and Stan, Stan provides an attractive software tool for moderately large networks with indirect and direct comparisons.