Exploring the Promise of Generative AI for Coding and Analyzing Qualitative Patient Data: Results From a Pilot Study in Non-Hodgkin’s Lymphoma
Author(s)
Karen Bailey, PhD1, Anne M. Skalicky, MPH2, Carla Dias Barbosa, MSc1, Meredith Y. Smith, MPA, PhD, FISPE1, Sonya L. Stanczyk, MPH1, Paul Cordero, PharmD, PhD3.
1PPD Evidera, Thermo Fisher Scientific, Wilmington, NC, USA, 2Senior Research Scientist, Thermo Fisher Scientific, Wilmington, NC, USA, 3Sanofi, London, United Kingdom.
1PPD Evidera, Thermo Fisher Scientific, Wilmington, NC, USA, 2Senior Research Scientist, Thermo Fisher Scientific, Wilmington, NC, USA, 3Sanofi, London, United Kingdom.
OBJECTIVES: Computer Assisted Qualitative Software (CAQDAS) programs have integrated machine learning and Artificial Intelligence (AI) into their tools with the promise of accelerating time-intensive processes related to qualitative data quality control, analysis and reporting. This study sought to understand how AI performs against human tasks/processes and provides recommendations for AI use-cases that can meet regulatory standards.
METHODS: Thirty one-on-one interviews were conducted to understand patient experience of Non-Hodgkin’s Lymphoma (NHL) and to evaluate patient-reported questionnaires that relate to their NHL condition. Interviews were conducted using a semi-structured interview guide, recorded and transcribed. Secondary analysis of study data with patient informed consent for secondary use of data and sponsor permission compared AI to human performance in creating codebooks, coding, analysing, and reporting. Four approaches using ATLAS.ti software evaluated AI compared to humans on the following: 1) traditional analysis using human creation of codebook and conduct of coding and analysis; 2) AI automated coding; 3) AI automated coding with human checks; and 4) AI intentional coding with the use of human input. The final outputs generated through these different approaches were compared.
RESULTS: Results highlight 1) the time and resources required for the completion of each approach; 2) comparison of themes and concepts identified using different approaches; 3) the balance of AI and human intelligence required to optimize outputs; and 4) use-case recommendations for ATLAS.ti AI adoption.
CONCLUSIONS: ATLAS.ti AI for qualitative data analysis has the potential to accelerate processes for conducting qualitative analysis but should include human oversight to ensure robust quality. Informed consent forms should describe use of AI, and analytic processes outlined in qualitative analysis plans. For more complex research questions, such as assessment of treatment benefit or meaningful within-patient change for clinical outcome assessments, further testing is warranted. Future work will should assess and incorporate regulatory and study sponsor guidelines and policies.
METHODS: Thirty one-on-one interviews were conducted to understand patient experience of Non-Hodgkin’s Lymphoma (NHL) and to evaluate patient-reported questionnaires that relate to their NHL condition. Interviews were conducted using a semi-structured interview guide, recorded and transcribed. Secondary analysis of study data with patient informed consent for secondary use of data and sponsor permission compared AI to human performance in creating codebooks, coding, analysing, and reporting. Four approaches using ATLAS.ti software evaluated AI compared to humans on the following: 1) traditional analysis using human creation of codebook and conduct of coding and analysis; 2) AI automated coding; 3) AI automated coding with human checks; and 4) AI intentional coding with the use of human input. The final outputs generated through these different approaches were compared.
RESULTS: Results highlight 1) the time and resources required for the completion of each approach; 2) comparison of themes and concepts identified using different approaches; 3) the balance of AI and human intelligence required to optimize outputs; and 4) use-case recommendations for ATLAS.ti AI adoption.
CONCLUSIONS: ATLAS.ti AI for qualitative data analysis has the potential to accelerate processes for conducting qualitative analysis but should include human oversight to ensure robust quality. Informed consent forms should describe use of AI, and analytic processes outlined in qualitative analysis plans. For more complex research questions, such as assessment of treatment benefit or meaningful within-patient change for clinical outcome assessments, further testing is warranted. Future work will should assess and incorporate regulatory and study sponsor guidelines and policies.
Conference/Value in Health Info
2025-11, ISPOR Europe 2025, Glasgow, Scotland
Value in Health, Volume 28, Issue S2
Code
PCR94
Topic
Methodological & Statistical Research, Patient-Centered Research
Topic Subcategory
Patient-reported Outcomes & Quality of Life Outcomes
Disease
No Additional Disease & Conditions/Specialized Treatment Areas, Oncology