Abstract

Objectives

This exploratory study aimed to develop a large language model (LLM)-based process to automate components of network meta-analysis (NMA), including model selection, analysis, output evaluation, and results interpretation. Automating these tasks with LLMs can enhance efficiency, consistency, and scalability in health economics and outcomes research, while ensuring that analyses adhere to established guidelines required by health technology assessment agencies. Improvements in efficiency and scalability may potentially become relevant as the European Union Health Technology Assessment Regulation comes into force, given anticipated analysis requirements and timelines.

Methods

Using Claude 3.5 Sonnet (V2), a process was designed to automate statistical model selection, NMA output evaluation, and results interpretation based on an “analysis-ready” data set. Validation was assessed by replicating examples from the National Institute for Health and Care Excellence Technical Support Document (TSD2), replicating results of non-Decision Support Unit-published NMAs, and generating comprehensive outputs (eg, heterogeneity, inconsistency, and convergence).

Results

The automated LLM-based process produced accurate results. Compared with TSD2 examples, differences were minimal, within expectations (given differences in sampling frameworks used), and comparable to those observed between estimates produced by the R vignettes against TSD2. Similar consistency was noted for non-Decision Support Unit-published NMA examples. Additionally, the LLM process generated and interpreted comprehensive NMA outputs.

Conclusions

This exploratory study demonstrates the feasibility of LLMs to automate key components of NMAs, determining the requisite NMA framework based only on input data. Further exploring these capabilities could clarify their role in streamlining NMA workflows.