OBJECTIVES: Network meta-analyses (NMA) are routinely used in health technology assessments (HTA) to assess the comparative efficacy, and safety, of treatments lacking head-to-head trials. The feasibility assessment (FA) stage, which involves synthesizing the numerous qualitative and quantitative data sources, can be time-consuming. With the rapid advancement of generative artificial intelligence (GenAI), there is potential to streamline the FA stage of the NMA process. This study investigates the effectiveness of Gemini 1.5 Pro (Gemini), a large language model (LLM) with a 1 million token context length, in expediting and enhancing the FA stage of an NMA.

METHODS: Gemini was used to reproduce the FAs for each of three NMA which were originally conducted in 2023-24. First, for each NMA, Gemini was prompted to identify and describe treatment effect modifiers (TEMs) and prognostic variables (PVs) based on the target population and indication. Next, the relevant publications for each NMA were uploaded to Gemini. Gemini was then prompted to create summary tables of baseline characteristics and population, interventions, comparators, outcomes, and study design (PICOS) information from the individual trials, based on the identified TEMs and PVs. Finally, Gemini was prompted to assess the heterogeneity of the trials and make recommendations for conducting the NMA with respect to statistical approaches. The tables and recommendations produced by Gemini were compared qualitatively to those produced manually.

RESULTS: Gemini correctly identified 50% of the TEMs and PVs. The produced tables were highly comparable to the manually produced results in completeness, accuracy, and variable categorization. Utilizing Gemini took approximately 3% of the time taken by the manual process.

CONCLUSIONS: Gemini was found to be able to expedite the FA process of an NMA effectively and efficiently, synthesizing and presenting results from diverse data sources, with time-saving capacities of nearly 97%.