RUNTIME COMPARISON OF BAYESIAN AND FREQUENTIST NETWORK META-ANALYSIS METHODS
Author(s)
Carla J. Pinkney, PhD1, Rebecca K. Judge, MSc2, Tristan Curteis, MSc1.
1Costello Medical, Manchester, United Kingdom, 2Costello Medical, London, United Kingdom.
1Costello Medical, Manchester, United Kingdom, 2Costello Medical, London, United Kingdom.
OBJECTIVES: Many software packages are available for implementation of Bayesian and Frequentist Network Meta-Analyses (NMAs) in R. Differences in computational performance are important, particularly given NMAs’ growing role in JCA and HTA to inform clinical and reimbursement decisions. This study investigated computational time and model output among popular R packages for NMAs.
METHODS: NMA models were fitted using the multinma, gemtc, nmaINLA and netmeta R packages, to assess the implementation of Stan-, JAGS- and INLA-based Bayesian and frequentist NMAs, respectively. All NMA models were fitted to a publicly available dataset, consisting of the mean off-time reduction in patients given dopamine agonists as adjunct therapy in Parkinson’s disease. Fixed and random effects models were fitted to the network of seven trials of four active drugs plus placebo. For the Stan- and JAGS-based methods, the number of sampling iterations per chain was set to satisfy convergence criteria, based on the Monte Carlo standard errors and Gelman-Rubin diagnostic. To investigate computational performance, runtimes were measured across 100 independent replicates (model fits) meeting the convergence criteria for each method.
RESULTS: Treatment rankings, point estimates and uncertainty intervals for relative treatment effects were similar across packages. With this small dataset and the random effect model, nmaINLA achieved the fastest mean runtime (2.11 seconds) compared with 6.82, 9.99 and 30.63 seconds for gemtc, netmeta and multinma respectively. The runtime for multinma was influenced by compilation overhead relative to its mean sampling time (7.57 seconds).
CONCLUSIONS: Of the software assessed, nmaINLA - a Bayesian approach which avoids iterative sampling - was the fastest. Although multinma offers more efficient sampling than gemtc, its compilation time was longer, meaning that gemtc and netmeta offered improved total runtimes in comparison with multinma. For larger datasets or more complicated models, the efficient sampling algorithms of Stan may outweigh the drawback of the multinma compilation time.
METHODS: NMA models were fitted using the multinma, gemtc, nmaINLA and netmeta R packages, to assess the implementation of Stan-, JAGS- and INLA-based Bayesian and frequentist NMAs, respectively. All NMA models were fitted to a publicly available dataset, consisting of the mean off-time reduction in patients given dopamine agonists as adjunct therapy in Parkinson’s disease. Fixed and random effects models were fitted to the network of seven trials of four active drugs plus placebo. For the Stan- and JAGS-based methods, the number of sampling iterations per chain was set to satisfy convergence criteria, based on the Monte Carlo standard errors and Gelman-Rubin diagnostic. To investigate computational performance, runtimes were measured across 100 independent replicates (model fits) meeting the convergence criteria for each method.
RESULTS: Treatment rankings, point estimates and uncertainty intervals for relative treatment effects were similar across packages. With this small dataset and the random effect model, nmaINLA achieved the fastest mean runtime (2.11 seconds) compared with 6.82, 9.99 and 30.63 seconds for gemtc, netmeta and multinma respectively. The runtime for multinma was influenced by compilation overhead relative to its mean sampling time (7.57 seconds).
CONCLUSIONS: Of the software assessed, nmaINLA - a Bayesian approach which avoids iterative sampling - was the fastest. Although multinma offers more efficient sampling than gemtc, its compilation time was longer, meaning that gemtc and netmeta offered improved total runtimes in comparison with multinma. For larger datasets or more complicated models, the efficient sampling algorithms of Stan may outweigh the drawback of the multinma compilation time.
Conference/Value in Health Info
2026-11, ISPOR Europe 2026, Vienna, Austria
Value in Health, Volume 29, Issue 12S
Code
SA36
Topic
Study Approaches
Topic Subcategory
Meta-Analysis & Indirect Comparisons
Disease
Neurological Disorders, No Additional Disease & Conditions/Specialized Treatment Areas