Leveraging Chat-GPT for Conducting Systematic Literature Reviews

OBJECTIVES: The objective of this study was to conduct a feasibility assessment of leveraging Chat-GPT for conducting Systematic Literature Reviews (SLRs).

METHODS: A modular SLR approach was designed to test various parts of the SLR process that could be leveraged and/or replaced with Chat-GPT. A custom Chat-GPT-4 application was developed in Python 3 to assist with (1) search strategy, (2) screening, (3) developing PRISMA diagrams, (4) developing summary tables, (5) developing datasets for meta-analysis, (6) conducting meta-analyses, (7) automating report generation, and (8) quality control checks. Each task was assessed for its potential to leverage Chat-GPT. Based on the testing and results, a new approach was developed for conducting SLRs.

RESULTS: For the eight SLR tasks, Chat-GPT showed varying levels of success. For the search strategy, Chat-GPT was highly useful in testing preliminary searches to create a set of terms with 100% accuracy. For screening, the results depended on the complexity of inclusion terms. For relatively simple inclusion criteria, Chat-GPT provided a 100% accurate set of included studies. However, for more complex criteria, screening proved challenging for Chat-GPT. PRISMA diagrams could be easily created with Chat-GPT. Summary tables were only partially created with Chat-GPT, as success was highly dependent on the design of templates and the type of outcomes needed in the tables. We had partial success in creating meta-analysis datasets using Chat-GPT. Report creation was technically feasible but lacked several insightful summaries. For quality control, Chat-GPT was highly useful and saved tedious proofreading efforts.

CONCLUSIONS: Systematic Literature Reviews vary in complexity. For simple SLRs, Chat-GPT can assist with many tasks. However, for more complex SLRs, Chat-GPT is only partially useful and should be used under the supervision of experienced SLR experts.