Session (CTI)

PURPOSE: This workshop will teach participants when and where to consider developing multi-agent AI systems in health economics and outcomes research (HEOR), what types of large language model (LLM) shortcomings these systems can address, and how to design multi-agent system architectures with clear understanding of objectives and design features. Participants will gain practical skills in transforming single-agent applications into multi-agent frameworks to improve quality while reducing subject matter expert burden, enabling immediate application to their own HEOR projects. DESCRIPTION: Single-agent AI systems in HEOR often require extensive human oversight due to quality and accuracy concerns, limiting practical adoption despite their promise. Multi-agent systems offer a solution by enabling AI-to-AI validation and correction before human review, significantly improving output reliability while reducing review time and costs. Dr. Caro will introduce the rapid integration of AI systems in HEOR activities and current quality challenges, establishing the rationale for the need to maintain research standards while reducing expert oversight burden (10 min). Mr. Deniz will present multi-agent system fundamentals, including when these systems make sense, key design considerations, and implementation strategies for HEOR applications (15 min). Dr. Reddy will demonstrate a real-world HEOR application currently using a single-agent structure, explaining its objectives, workflow, and limitations (15 min). The workshop will conclude with an interactive group exercise where participants will collectively transform the presented single-agent system into a multi-agent architecture, applying the design principles discussed earlier. As a group, attendees will discuss specific validation checkpoints, define agent roles and responsibilities, interaction protocols, and develop quality assurance mechanisms (20 min). This hands-on approach ensures participants leave with practical experience in multi-agent system design and actionable frameworks they can implement in their own HEOR applications.