REPORTING AND INTERPRETATION OF COGNITIVE DEBRIEFING RESULTS IN PATIENT REPORTED OUTCOME MEASURE VALIDATION: ARE THERE STANDARDS?
Author(s)
Nathan Johnson, MPH, Martha Gauthier, MA, Julie Lynn Whyte, MPH.
Patient Centered Outcomes, Lumanity, Boston, MA, USA.
Patient Centered Outcomes, Lumanity, Boston, MA, USA.
OBJECTIVES: Cognitive debriefing (CD) studies are conducted to assess whether patients interpret patient-reported outcome (PRO) measures as intended, in accordance with Food and Drug Administration guidance and ISPOR task force recommendations. However, there is no clear guidance on what constitutes an acceptable level of understanding. This study aimed to examine how CD results have been categorized in published research and to determine whether standard quantitative thresholds or cutoffs are used in their interpretation.
METHODS: We conducted a targeted review of the ISPOR and ISOQOL conference abstract databases. The search terms “cognitive debriefing” and “content validation” were used to identify abstracts that discussed CD thresholds, either as cutoffs for interpretation or as recommendations. Eligible abstracts described CD studies of PRO measures. Each abstract was reviewed to determine whether CD results were reported quantitatively and whether a predefined threshold or cutoff was specified for interpreting participant understanding. Any identified cutoffs were recorded.
RESULTS: Few abstracts reported CD results quantitatively (e.g., the proportion of participants interpreting items as intended). Among those that did, explicit thresholds were rare. When thresholds were mentioned, they varied considerably (e.g., ≥80% item comprehension) and were typically applied descriptively rather than as a formal decision rule. No consistent or widely endorsed standard for acceptable comprehension emerged across studies.
CONCLUSIONS: This review found limited transparency and consistency in the reporting and interpretation of CD results in PRO measure content validation. While some studies apply quantitative thresholds, these are neither common nor standardized, and often lack explicit justification. Interpretation of CD findings is nuanced and should consider study context, target population, and item complexity. Standardized reporting guidelines with recommended threshold ranges could improve comparability and methodological rigor in CD studies.
METHODS: We conducted a targeted review of the ISPOR and ISOQOL conference abstract databases. The search terms “cognitive debriefing” and “content validation” were used to identify abstracts that discussed CD thresholds, either as cutoffs for interpretation or as recommendations. Eligible abstracts described CD studies of PRO measures. Each abstract was reviewed to determine whether CD results were reported quantitatively and whether a predefined threshold or cutoff was specified for interpreting participant understanding. Any identified cutoffs were recorded.
RESULTS: Few abstracts reported CD results quantitatively (e.g., the proportion of participants interpreting items as intended). Among those that did, explicit thresholds were rare. When thresholds were mentioned, they varied considerably (e.g., ≥80% item comprehension) and were typically applied descriptively rather than as a formal decision rule. No consistent or widely endorsed standard for acceptable comprehension emerged across studies.
CONCLUSIONS: This review found limited transparency and consistency in the reporting and interpretation of CD results in PRO measure content validation. While some studies apply quantitative thresholds, these are neither common nor standardized, and often lack explicit justification. Interpretation of CD findings is nuanced and should consider study context, target population, and item complexity. Standardized reporting guidelines with recommended threshold ranges could improve comparability and methodological rigor in CD studies.
Conference/Value in Health Info
2026-05, ISPOR 2026, Philadelphia, PA, USA
Value in Health, Volume 29, Issue S6
Code
PCR39
Topic
Patient-Centered Research
Topic Subcategory
Instrument Development, Validation, & Translation
Disease
No Additional Disease & Conditions/Specialized Treatment Areas