OBJECTIVES: The number of software packages available to conduct network meta-analysis (NMA) is increasing, potentially offering gains in computation time, model convergence, and ease of use. This case study assessed the differences between Stan and WinBUGS for conducting a hazard ratio (HR) NMA.

METHODS: An HR NMA was conducted using WinBUGS run via R2WinBUGS and Stan run via multinma statistical software. A previously published network of evidence for overall survival (OS) of patients on first-line treatment for renal cell carcinoma (RCC) was replicated for this case study. A fixed effects model was fitted to the star-shaped network with 6 trials and 7 treatments. The following performance indicators for the software packages were compared: differences in numerical modelled outcomes, including point estimates (HRs) and 95% credible intervals (CrI); computation time; model convergence and user experience; trends in residual deviance for each datapoint.

RESULTS: Stan and WinBUGS yielded similar numerical outcomes: differences in point were minor for all comparisons (<0.001). The estimates for nivolumab + ipilimumab versus sunitinib were 0.690 using Stan (CrI 0.589-0.809) versus 0.690 using WinBUGS (CrI 0.590-0.809). Both software packages produced comparable results in the treatment rankings, the credible intervals of the estimates and the interpretation. Stan runs via multinma provided improved user experience with shorter computation time, more informative error messages and easier assessment of model convergence than WinBUGS runs.

CONCLUSIONS: Our case study suggests that comparable results and interpretations can be derived when conducting an HR based NMA with Stan or WinBUGS in a straightforward evidence network. Stan provided advantages in computation time and overall user experience when facing errors and model convergence queries.