Presentation (CTI)

OBJECTIVES: Health economic modeling is critical for healthcare decision-making amongst that model conceptualization remains resource-intensive and time-consuming, particularly for rare diseases with limited evidence. This study demonstrates the application of multi-agentic generative AI (GenAI) systems using Retrieval-Augmented Generation (RAG) to develop comprehensive model protocols for health technology assessment (HTA) submissions, addressing two distinct scenarios: de novo model development in evidence-scarce therapeutic areas and model enhancement where existing structures exist.
METHODS: A multi-agentic AI framework was developed with specialized agents representing clinicians, statisticians and health economists. For de novo model conceptualization, system leveraged key opinion leader (KOL) inputs from meeting notes, transcripts, and Large Language Models (LLM) training data. For therapeutic areas with existing models, published evidence from peer-reviewed literature and previous HTA submissions were utilized through RAG architecture. System was designed to generate model structures including health states and transitions, identified key assumptions, specified model inputs (clinical parameters, costs, health-related quality of life values), and recommended relevant sensitivity and scenario analyses. Interactive feedback loops were incorporated to iteratively refine protocol outputs. Generated protocols underwent validation through clinical expert review and comparison against gold-standard manual protocols.
RESULTS: The multi-agentic system produced HTA-ready protocols with 90% overall alignment with subject matter expert assessments. Development time was reduced by 85% compared to traditional manual workflows resulting in time and resource savings. Section-specific accuracy varied by complexity: cost inputs achieved 96% accuracy, utility inputs showed 90% accuracy, while model structure and clinical pathway conceptualization demonstrated 85% accuracy (95% where model already exists and 80% in de-novo model conceptualization), reflecting the inherent complexity of de novo conceptualization.
CONCLUSIONS: Multi-agentic AI systems significantly accelerate model protocol development while maintaining methodological rigor, particularly valuable for rare diseases with limited evidence. Future development will focus on extending capabilities to discrete event simulation models, agent-based models, and compartmental mathematical models for infectious disease.