Structural Uncertainty in Evidence Synthesis: A Case Study Applying Model Averaging in Bayesian Multi-Level Network Meta-Regression

OBJECTIVES: Bayesian model averaging (BMA) has been proposed in the context of economic modeling to address structural uncertainty when multiple survival extrapolations are clinically plausible. With the growing number of modeling approaches for (network) meta-analysis of survival data, we sought to capture structural uncertainty in evidence synthesis. Using a case study in newly diagnosed multiple myeloma (ndMM), we implemented model averaging across multi-level meta-regression (ML-NMR) models.

METHODS: A previously published network of treatments for ndMM using constructed synthetic data was used. The evidence consisted of 5 trials having individual patient-level data (IPD) or aggregate-level pseudo-IPD for progression-free survival. Treatments were: lenalidomide (len), thalidomide (thal), and placebo/observation. We fit ML-NMR models using 9 likelihoods: log-normal, log-logistic, Weibull (proportional hazards [PH] and accelerated failure time), M-spline, Gompertz, exponential, gamma, and generalized gamma. Within 5 target populations defined by each trial, relative treatment effects for restricted mean survival time (∆RMST) were averaged using: pseudo-BMA weights using Pareto smoothed importance-sampling leave-one-out cross-validation (PSIS-LOO); and Bayesian stacking of predictive distributions.

RESULTS: Amongst the 9 models, 2 did not converge, 3 had negligible weights for both averaging methods, and 1 had negligible weights for the pseudo-BMA method only. The largest variation in ∆RSMT (at 48-months) between the candidate models was for the thal-len comparison; model-specific estimates ranged from -5.86 months (log-normal; 95% credible interval [CrI]: -9.40, -2.60) to -4.24 months (Weibull PH; 95% CrI: -6.84, -1.88). Model-averaged estimates encompassed this structural uncertainty, yielding estimates of -5.73 months (95% CrI: -9.35, -2.55) with pseudo-BMA weights and -5.41 months (95% CrI: -9.14, -2.31) with stacking.

CONCLUSIONS: This example of model averaging demonstrates a method to account for structural uncertainty in survival-based ML-NMR, and can be applied in other meta-analysis settings. Further research could extend this work by incorporating external information (to inform clinical plausibility of model extrapolations) into the model averaging framework.