OBJECTIVES: The advancement of Large Language Models (LLMs), such as GPT-4, provides opportunities for automating data extraction and analysis in systematic reviews and meta-analyses. However, their practical application in Health Economics and Outcomes Research (HEOR) remains unverified. Our study aimed to evaluate GPT-4's accuracy in replicating a Network Meta-Analysis (NMA) result on overall survival of adult patients with advanced non-small cell lung cancer (NSCLC) post platinum-based treatment and pre-immunotherapy.

METHODS: Using GPT-4 through a Python API, we extracted survival data from the abstracts of eleven studies, transformed it to the log scale, and generated an R script for NMA using generic code from the R 'multinma' package. GPT-4 updated the code with new data and produced an executable R script that was run end-to-end in Docker, and parsed Docker output to create a mini NMA report. This was compared to the original human conducted NMA.

RESULTS: The LLM-generated model accurately replicated the overall survival outcome using abstracts and generic R code. It successfully extracted and converted survival data, created the NMA R script, ran it using Docker, and accurately produced the original NMA results. This was achieved with a single generic python script, demonstrating GPT-4's capability to perform end-to-end NMA using unstructured abstracts.

CONCLUSIONS: This study offers promising evidence for the potential of AI models like GPT-4 in automating data extraction and NMA. Further studies are necessary to confirm these findings in diverse contexts and investigate AI's potential in enhancing systematic reviews and NMA. Further exploration is also required on multimodal versions, and the ability of LLMs to validate the proportional hazards assumption.