Session (CTI)

Large Language Models are powerful tools that have transformed how we interact with information, but foundational models are not designed for conducting scientific research. Given the critical implications of HEOR research on healthcare decision-making, understanding both the capabilities and limitations of these LLM is paramount for their effective use. This session presents a structured approach, divided into two parts. The first part examines the fundamental capabilities and limitations of LLMs in HEOR contexts, discussing critical considerations such as data quality, reproducibility, and validation requirements. The second part features an interactive demonstration comparing basic LLM usage versus advanced research design through a real-world example of disease burden analysis. Audience members will help define the research context, allowing for immediate illustration of key concepts. This session will particularly benefit researchers, analysts, and decision-makers who are interested in leveraging AI tools while maintaining scientific integrity in their work. Participants will leave with practical knowledge of how to design and implement LLM-enhanced research workflows that meet HEOR's rigorous methodological standards. Proposed Session Flow (rough timing): Introduction (5 mins) Part 1: LLMs in HEOR Research (15 mins) - Core capabilities and limitations - Specific challenges in HEOR applications - Best practices and safeguards Part 2: Interactive Demonstration (25 mins) - Case study: Disease burden analysis - Audience input on context/parameters - Comparison of: - Basic LLM approach (direct ChatGPT use) - Advanced research design (structured prompts, function calls, etc.) - Discussion of quality differences and implications Q&A (15 mins)