Fixed Effect Versus Random Effects Bayesian Network Meta-Analyses in Practice: What Is Used to Inform National Institute for Health and Care Excellence (NICE) Technology Appraisals?

OBJECTIVES: The literature provides clear guidance on when the use of fixed effect (FE) and random effects (RE) assumptions in network meta-analysis (NMA) are appropriate. However, practice may differ. This study reviewed documents published by NICE to assess the application of FE and RE assumptions, the rationale surrounding this choice, consideration of alternative priors, and decision maker feedback, with the aim to determine which assumption was preferred.

METHODS: All technology appraisals (TAs) published on the NICE website between were included. Requests for unavailable supporting documents were not made. Only single TAs with an indirect treatment comparison (ITC) without changes to patient access schemes, market authorisation, or recommendations were considered.

RESULTS: Of the 257 TAs screened, 123 contained at least one ITC, of which 64 included a Bayesian NMA. Of these, 53 (83%) included FE and 49 (77%) included RE NMAs; 6 (9%) did not specify. For the 49 RE NMAs, 31 (63%) specified the prior used for the between-study standard deviation; 17 (55%) submissions used noninformative priors, 10 (32%) submissions used informative priors and 4 (13%) used both. Most submissions (45 [70%]) tested both FE and RE, and 30 (66%) of these preferred a FE NMA, frequently justified by sparse networks, lack of RE convergence or improbably large credible intervals. This contrasted with Evidence Review Group opinion, which tended to prefer RE NMAs, explicitly recommending informative priors when RE NMAs did not converge.

CONCLUSIONS: In submissions to NICE, FE NMAs tend to be favoured in networks with limited data availability or low heterogeneity, and RE NMAs otherwise. Informative priors were recommended to address convergence issues in RE NMAs. Guidance is usually followed with both FE and RE NMAs being tested, but vague priors tend to be used over informative priors, which may impact convergence.