Abstract

Objectives

Artificial intelligence (AI) is widely used in healthcare for various purposes, with generative AI (GAI) increasingly being applied to systematic review (SR) processes. We aimed to summarize the evidence on the performance metrics of GAI in the SR process.

Methods

PubMed, EMBASE, Scopus, and ProQuest Dissertations Theses Global were searched from their inception up to March 2025. Only experimental studies that compared GAI with other GAIs or human reviewers at any stage of the SR were included. Modified Quality Assessment of Diagnostic Accuracy Studies version 2 was used to assess the quality of the studies that used GAI in the study selection process. We summarized the findings of the included studies using a narrative approach.

Results

Out of 7418 records screened, 30 studies were included. These studies used GAI tools such as ChatGPT, Bard, and Microsoft Bing AI. GAI appears to be effective for participant, intervention, comparator, and outcome formulation and data extraction processes, including complex information. However, because of inconsistent reliability, GAI is not recommended for literature search and study selection as it may retrieve nonrelevant articles and yield inconsistent results. There was mixed evidence on whether GAI can be used for risk of bias assessment. Studies using GAI for study selection were generally of high quality based on the modified Quality Assessment of Diagnostic Accuracy Studies version 2.

Conclusions

GAI shows promising support in participant, intervention, comparator, and outcome-based question formulation and data extraction. Although it holds potential to enhance the SR process in healthcare, further practical application and validated evidence are needed before it can be fully integrated into standard workflows.