A Retrieval-Augmented Generation (RAG)-Based Framework to Automate Report Writing for Systematic Literature Reviews (SLRs) in Evidence Synthesis
Author(s)
Gagandeep Kaur, MPharm, Ankita Sood, MPH, PharmD, Vedant Soni, B.Tech, Rajdeep Kaur, Shubhram Pandey, MSc, Barinder Singh, RPh.
Pharmacoevidence Pvt. Ltd., Mohali, India.
Pharmacoevidence Pvt. Ltd., Mohali, India.
OBJECTIVES: The regulatory agencies acknowledge the potential of artificial intelligence (AI) to enhance evidence generation in SLRs, while underscoring the need for continued human oversight. This study aimed to evaluate the use of a RAG framework to generate evidence-linked reports for SLRs in alignment with regulatory guidelines.
METHODS: An integrated framework combining RAG processing pipeline with the multi agentic approach was used to generate the structured outputs. AI agents were developed to generate specific sections of an SLR on Huntington’s disease, focusing on areas like humanistic burden, economic burden, and unmet needs from the uploaded documents. The output was generated in a Word template and subsequently assessed by subject matter experts (SMEs) through a 5-point Likert scale. The scale included: Strongly Agree (factually accurate and fully traceable to evidence), Agree (minor edits required; evidence clearly traceable), Neutral (mostly accurate; though some terminology lacked clear sourcing), Disagree (factual inconsistencies or incomplete evidence linkage), and Strongly Disagree (inaccurate content with missing or unverifiable evidence).
RESULTS: All outputs related to humanistic burden were rated as “Strongly agree”. For economic burden and unmet needs, most content received “strongly agree” or “agree”, with only two sentences requiring minor terminology edits. All generated tables and figures were accurate, except for one instance where a calculated value lacked traceability to the original source, resulting in a “disagree” rating. Overall, the RAG-enabled framework supported successful report generation with minimal SME intervention and demonstrated a significant time saving of approximately 75%.
CONCLUSIONS: This study demonstrates that a RAG-based generative AI framework can effectively support SLR writing. Through structured SME validation and minimal human oversight, the approach enables efficient, compliant content generation, supporting responsible AI integration in health economics and outcomes research.
METHODS: An integrated framework combining RAG processing pipeline with the multi agentic approach was used to generate the structured outputs. AI agents were developed to generate specific sections of an SLR on Huntington’s disease, focusing on areas like humanistic burden, economic burden, and unmet needs from the uploaded documents. The output was generated in a Word template and subsequently assessed by subject matter experts (SMEs) through a 5-point Likert scale. The scale included: Strongly Agree (factually accurate and fully traceable to evidence), Agree (minor edits required; evidence clearly traceable), Neutral (mostly accurate; though some terminology lacked clear sourcing), Disagree (factual inconsistencies or incomplete evidence linkage), and Strongly Disagree (inaccurate content with missing or unverifiable evidence).
RESULTS: All outputs related to humanistic burden were rated as “Strongly agree”. For economic burden and unmet needs, most content received “strongly agree” or “agree”, with only two sentences requiring minor terminology edits. All generated tables and figures were accurate, except for one instance where a calculated value lacked traceability to the original source, resulting in a “disagree” rating. Overall, the RAG-enabled framework supported successful report generation with minimal SME intervention and demonstrated a significant time saving of approximately 75%.
CONCLUSIONS: This study demonstrates that a RAG-based generative AI framework can effectively support SLR writing. Through structured SME validation and minimal human oversight, the approach enables efficient, compliant content generation, supporting responsible AI integration in health economics and outcomes research.
Conference/Value in Health Info
2025-11, ISPOR Europe 2025, Glasgow, Scotland
Value in Health, Volume 28, Issue S2
Code
MSR9
Topic
Methodological & Statistical Research
Topic Subcategory
Artificial Intelligence, Machine Learning, Predictive Analytics
Disease
No Additional Disease & Conditions/Specialized Treatment Areas