Use of AI Chat in Improving Qualitative Interview Guide Clarity

OBJECTIVES: To examine artificial intelligence (AI) chat’s ability to act as a mock interview participant to pilot and refine qualitative interview guides prior to use with interview participants.

METHODS: AI chat was prompted to pretend to be a 6^th grader or 3^rd grader with fatigue who was participating in a qualitative interview. Interview questions followed the structure of a cognitive debriefing interview, presenting the “interviewee” with an example item from the PROMIS Pediatric Fatigue – Short Form 10a (“I was so tired it was hard for me to pay attention”) and asking questions to evaluate the understandability, clarity, and relevance of the item, as well as meaningful change in the concept. AI chat’s responses were evaluated to help determine the appropriateness of question wording in the interview guide and collect possible rewording of the interview probes to improve clarity.

RESULTS: The AI chat mock interview participant responded to questions about interpretation and clarity of the item and response options with relevant reference to the experiences and expected literacy of a 6^th grader and 3^rd grader. AI chat gave responses that were realistic for a child experiencing fatigue. Upon providing what would be considered accurate responses to questions, AI chat was also prompted if improvements could be made to questions in the interview guide. AI chat provided suggestions for revised question wording that maintained the original intention of the item but potentially felt more natural and relatable to a 6^th grader or 3^rd grader.

CONCLUSIONS: AI chat may be a useful tool for piloting and refining interview guide question wording and/or practicing interview techniques. However, it does not replace the experience of engaging with a participant directly, particularly a child participant, or help to anticipate variability in individual interview participants in a specific population or disease area.