Session (CTI)

PURPOSE: As HEOR embraces generative AI (GenAI) to improve efficiency, an important paradox has emerged. GenAI adoption overwhelmingly targets structured computational tasks (modeling, network meta-analysis, systematic literature reviews), with supporting ISPOR guidance: ELEVATE-AI and CHEERS-AI. However, tasks using largely unstructured qualitative data remain underexplored despite GenAI's potential to excel in critical HEOR workflows: synthesizing evidence repositories, analyzing patient experience data (PED), including patient interviews or social media data sources, and informing study design. This workshop shares validation approaches for GenAI applications in qualitative evidence generation and summarization. DESCRIPTION: Katelyn Keyloun (Arysana) will provide a 5-minute overview describing emerging validation approaches for probabilistic Large Language Models (LLMs) using largely qualitative unstructured data. The audience will explore new approaches from Panelists representing multidisciplinary perspectives, who will present real-world applications (30 minutes total) describing how AI processes, analyzes, and summarizes unstructured qualitative data, emphasizing validation approaches: 1. Conversational AI for Data Collection (Bill Byrom, Signant Health): LLM-powered chatbots conducting qualitative in-trial patient interviews, including validation of interview technique quality, transcript generation, and reporting capabilities versus traditional human approaches. 2. Generate Early Patient Insights from Social Media (Catherine Foley, AbbVie): Leveraging GenAI to extract and synthesize patient perspectives from digital communities, addressing validation challenges for patient-generated content and early signal detection. 3. LLMs for Study Design (Justyna Amelio, AbbVie): Applying GenAI to inform study design decisions and protocol development by synthesizing evidence across multiple sources, and validating AI-generated recommendations. Interactive audience participation (20 minutes) includes polls and moderated discussion exploring opportunities in adopting GenAI, including validation, implementation barriers, and organizational readiness.