Leveraging Chat-GPT for Conducting Systematic Literature Reviews

Speaker(s)

Aggarwal S1, Kumar S2, Topaloglu O3
1NOVEL Health Strategies, Chevy Chase, MD, USA, 2NOVEL HEALTH STRATEGIES, COLUMBIA, MD, USA, 3NOVEL Health Strategies, Bethesda, MD, USA

OBJECTIVES: The objective of this study was to conduct a feasibility assessment of leveraging Chat-GPT for conducting Systematic Literature Reviews (SLRs).

METHODS: A modular SLR approach was designed to test various parts of the SLR process that could be leveraged and/or replaced with Chat-GPT. A custom Chat-GPT-4 application was developed in Python 3 to assist with (1) search strategy, (2) screening, (3) developing PRISMA diagrams, (4) developing summary tables, (5) developing datasets for meta-analysis, (6) conducting meta-analyses, (7) automating report generation, and (8) quality control checks. Each task was assessed for its potential to leverage Chat-GPT. Based on the testing and results, a new approach was developed for conducting SLRs.

RESULTS: For the eight SLR tasks, Chat-GPT showed varying levels of success. For the search strategy, Chat-GPT was highly useful in testing preliminary searches to create a set of terms with 100% accuracy. For screening, the results depended on the complexity of inclusion terms. For relatively simple inclusion criteria, Chat-GPT provided a 100% accurate set of included studies. However, for more complex criteria, screening proved challenging for Chat-GPT. PRISMA diagrams could be easily created with Chat-GPT. Summary tables were only partially created with Chat-GPT, as success was highly dependent on the design of templates and the type of outcomes needed in the tables. We had partial success in creating meta-analysis datasets using Chat-GPT. Report creation was technically feasible but lacked several insightful summaries. For quality control, Chat-GPT was highly useful and saved tedious proofreading efforts.

CONCLUSIONS: Systematic Literature Reviews vary in complexity. For simple SLRs, Chat-GPT can assist with many tasks. However, for more complex SLRs, Chat-GPT is only partially useful and should be used under the supervision of experienced SLR experts.

Code

MSR174

Topic

Methodological & Statistical Research, Study Approaches

Topic Subcategory

Artificial Intelligence, Machine Learning, Predictive Analytics, Literature Review & Synthesis

Disease

No Additional Disease & Conditions/Specialized Treatment Areas