AI-Driven Evidence Synthesis: Leveraging RAG and Multi-Agentic Approach to Conduct Disease Landscape Assessments
Moderator
Shubhram Pandey, MSc, Pharmacoevidence Pvt. Ltd., SAS Nagar, Mohali, India
Speakers
Kasper Johannesen, PhD; Barinder Singh, RPh, Pharmacoevidence Pvt. Ltd., SAS Nagar Mohali, India; Rajdeep Kaur, PhD, Pharmacoevidence Pvt. Ltd., Mohali, India; Vicky Huang; Sven L Klijn, MSc, Bristol Myers Squibb, Princeton, NJ, United States; Christian Ernst Heinrich Boehler, MSc, PhD; Lisa Vaz; Matthew Walters; Nicola Waddell, Pharmacoevidence Pvt. Ltd., London, United Kingdom
OBJECTIVES: Health Technology Assessment (HTA) plays a pivotal role in evaluating the value of medical interventions. The objective of this study was to develop an AI powered platform for conducting landscape assessments of HTA data, including treatment guidelines, regulatory information and HTA assessments across different disease areas.
METHODS: An AI-driven interface was built using Python microservices and a RAG-based data processing pipeline. This data pipeline enables users to upload HTA relevant information for various disease areas. The data processing pipeline standardizes the data into markdown format using Optical Character Recognition (OCR). The markdown data is further divided into small chunks, and embeddings with metadata are stored in a vector database. LangChain-based AI agents work throughout the pipeline to provide precise evidence retrieval. Validation was conducted by domain experts leveraging a 5-point Likert scale to evaluate the AI-generated output sets in response to 30 complex prompts.
RESULTS: The interface effectively automated the evidence synthesis process from HTA data uploaded to the RAG engine, significantly reducing the time required to extract relevant data. Key features include intervention and country-wise reimbursement status and their key drivers, categorization of clinical, economic, and patient reported outcomes, and their integration into decision support frameworks. Domain experts judged that 27/30 of AI-generated output sets (90.0%) demonstrated strong alignment with human knowledge, 2/30 (6.7%) showed alignment and 1/30 (3.3%) was undecided.
CONCLUSIONS: The platform constitutes a major step forward in automating and simplifying the evidence synthesis process using generative AI approaches and the RAG pipeline. Pilot testing demonstrated that the tool can process complex HTA data from multiple sources and therapeutic areas. The automated approach also ensures scalability, making it adaptable to various disease areas and project scopes. The platform has the potential to significantly reduce the time to extract the insights from HTA data with high accuracy.
METHODS: An AI-driven interface was built using Python microservices and a RAG-based data processing pipeline. This data pipeline enables users to upload HTA relevant information for various disease areas. The data processing pipeline standardizes the data into markdown format using Optical Character Recognition (OCR). The markdown data is further divided into small chunks, and embeddings with metadata are stored in a vector database. LangChain-based AI agents work throughout the pipeline to provide precise evidence retrieval. Validation was conducted by domain experts leveraging a 5-point Likert scale to evaluate the AI-generated output sets in response to 30 complex prompts.
RESULTS: The interface effectively automated the evidence synthesis process from HTA data uploaded to the RAG engine, significantly reducing the time required to extract relevant data. Key features include intervention and country-wise reimbursement status and their key drivers, categorization of clinical, economic, and patient reported outcomes, and their integration into decision support frameworks. Domain experts judged that 27/30 of AI-generated output sets (90.0%) demonstrated strong alignment with human knowledge, 2/30 (6.7%) showed alignment and 1/30 (3.3%) was undecided.
CONCLUSIONS: The platform constitutes a major step forward in automating and simplifying the evidence synthesis process using generative AI approaches and the RAG pipeline. Pilot testing demonstrated that the tool can process complex HTA data from multiple sources and therapeutic areas. The automated approach also ensures scalability, making it adaptable to various disease areas and project scopes. The platform has the potential to significantly reduce the time to extract the insights from HTA data with high accuracy.
Conference/Value in Health Info
2025-05, ISPOR 2025, Montréal, Quebec, CA
Value in Health, Volume 28, Issue S1
Code
MSR16
Topic
Methodological & Statistical Research
Topic Subcategory
Artificial Intelligence, Machine Learning, Predictive Analytics
Disease
No Additional Disease & Conditions/Specialized Treatment Areas