Presentation (CTI)

OBJECTIVES: Artificial intelligence offers potential to streamline thematic analysis of qualitative data in stakeholder research for health economics outcomes research (HEOR), a process which has traditionally been time and resource intensive. This study explores how large language models (LLM) can improve the efficiency of thematic analyses whilst maintaining analytical accuracy. It also examines the role of expert researchers in guiding and interpreting AI-generated outputs to ensure contextual depth.
METHODS: Six interview transcripts from healthcare professionals, focusing on unmet needs within a haematological disease area, were thematically analysed using ChatGPT-4.1 as part of an internal research initiative. Researchers guided the LLM iteratively through prompt development and output refinement. LLM-generated themes were reviewed against manual analysis for alignment and accuracy. To explore scalability, the LLM was also tested on 20 transcripts.
RESULTS: The LLM accurately identified key themes such as epidemiology, clinical burden, and treatment pathways, with minimal revision. However, it struggled to interpret nuanced language conflating opposing mortality expectations and overlooking distinct psychosocial impacts. This was resolved through researcher prompting. When scaled to 20 transcripts, the model necessitated segmentation to manage input length. This segmentation, however, introduced hallucinations and inconsistencies, raising concerns regarding the model’s ability to synthesise large qualitative datasets.
CONCLUSIONS: LLM can accelerate thematic analysis and facilitate the integration of key stakeholder insights into evidence submissions, potentially expediting evidence generation. However, the LLM struggled to capture the nuances of human connectivity, producing more basic outputs that lack depth, and struggles to synthesise large datasets without expert oversight. Human expertise remains essential to prompt refinement and contextual interpretation. As AI models evolve, integration within qualitative research may offer meaningful opportunities to streamline analysis. However, it remains important to implement with structured guidance and expert oversight. Future research should focus on developing scalable approaches to maintain accuracy across larger transcript sets.