AI-Driven Evidence Synthesis: Leveraging RAG and Multi-Agentic Approach to Conduct Disease Landscape Assessments
Author(s)
Kasper Munk Johannesen, PhD, MSc1, Barinder Singh, RPh2, Rajdeep Kaur, PhD3, Vicky Huang, MSc4, Sven L. Klijn, MSc4, Christian Ernst Heinrich Boehler, MSc, PhD5, Lisa Vaz, MBA4, Matthew Walters, BA Hons6, Nicola H. Waddell, HNC2, Shubhram Pandey, MSc3.
1Bristol Myers Squibb, Stockholm, Sweden, 2Pharmacoevidence, London, United Kingdom, 3Pharmacoevidence, Mohali, India, 4Bristol Myers Squibb, Princeton, NJ, USA, 5Bristol Myers Squibb, Vienna, Austria, 6Bristol Myers Squibb, Uxbridge, United Kingdom.
1Bristol Myers Squibb, Stockholm, Sweden, 2Pharmacoevidence, London, United Kingdom, 3Pharmacoevidence, Mohali, India, 4Bristol Myers Squibb, Princeton, NJ, USA, 5Bristol Myers Squibb, Vienna, Austria, 6Bristol Myers Squibb, Uxbridge, United Kingdom.
Presentation Documents
OBJECTIVES: Health Technology Assessment (HTA) plays a pivotal role in evaluating the value of medical interventions. The objective of this study was to develop an AI powered platform for conducting landscape assessments of HTA data, including treatment guidelines, regulatory information and HTA assessments across different disease areas.
METHODS: An AI-driven interface was built using Python microservices and a RAG-based data processing pipeline. This data pipeline enables users to upload HTA relevant information for various disease areas. The data processing pipeline standardizes the data into markdown format using Optical Character Recognition (OCR). The markdown data is further divided into small chunks, and embeddings with metadata are stored in a vector database. LangChain-based AI agents work throughout the pipeline to provide precise evidence retrieval. Validation was conducted by domain experts leveraging a 5-point Likert scale to evaluate the AI-generated output sets in response to 30 complex prompts.
RESULTS: The interface effectively automated the evidence synthesis process from HTA data uploaded to the RAG engine, significantly reducing the time required to extract relevant data. Key features include intervention and country-wise reimbursement status and their key drivers, categorization of clinical, economic, and patient reported outcomes, and their integration into decision support frameworks. Domain experts judged that 27/30 of AI-generated output sets (90.0%) demonstrated strong alignment with human knowledge, 2/30 (6.7%) showed alignment and 1/30 (3.3%) was undecided.
CONCLUSIONS: The platform constitutes a major step forward in automating and simplifying the evidence synthesis process using generative AI approaches and the RAG pipeline. Pilot testing demonstrated that the tool can process complex HTA data from multiple sources and therapeutic areas. The automated approach also ensures scalability, making it adaptable to various disease areas and project scopes. The platform has the potential to significantly reduce the time to extract the insights from HTA data with high accuracy.
METHODS: An AI-driven interface was built using Python microservices and a RAG-based data processing pipeline. This data pipeline enables users to upload HTA relevant information for various disease areas. The data processing pipeline standardizes the data into markdown format using Optical Character Recognition (OCR). The markdown data is further divided into small chunks, and embeddings with metadata are stored in a vector database. LangChain-based AI agents work throughout the pipeline to provide precise evidence retrieval. Validation was conducted by domain experts leveraging a 5-point Likert scale to evaluate the AI-generated output sets in response to 30 complex prompts.
RESULTS: The interface effectively automated the evidence synthesis process from HTA data uploaded to the RAG engine, significantly reducing the time required to extract relevant data. Key features include intervention and country-wise reimbursement status and their key drivers, categorization of clinical, economic, and patient reported outcomes, and their integration into decision support frameworks. Domain experts judged that 27/30 of AI-generated output sets (90.0%) demonstrated strong alignment with human knowledge, 2/30 (6.7%) showed alignment and 1/30 (3.3%) was undecided.
CONCLUSIONS: The platform constitutes a major step forward in automating and simplifying the evidence synthesis process using generative AI approaches and the RAG pipeline. Pilot testing demonstrated that the tool can process complex HTA data from multiple sources and therapeutic areas. The automated approach also ensures scalability, making it adaptable to various disease areas and project scopes. The platform has the potential to significantly reduce the time to extract the insights from HTA data with high accuracy.
Conference/Value in Health Info
2025-05, ISPOR 2025, Montréal, Quebec, CA
Value in Health, Volume 28, Issue S1
Code
MSR16
Topic
Methodological & Statistical Research
Topic Subcategory
Artificial Intelligence, Machine Learning, Predictive Analytics
Disease
No Additional Disease & Conditions/Specialized Treatment Areas