EXPLORING CONTENT VALIDITY OF GENERIC PREFERENCE-WEIGHTED MEASURES: A SCOPING REVIEW
Author(s)
Jill Carlton, PhD1, Tessa Peasgood, PhD1, Anthea Sutton, BA,MA2, Sarah Derrett, PhD3, Janine Verstraete, PhD4, Brendan Mulhern, PhD5, Nan Luo, PhD6.
1SCHARR, University of Sheffield, Sheffield, United Kingdom, 2ScHARR, University of Sheffield, Sheffield, United Kingdom, 3University of Otago, Dunedin, New Zealand, 4University of Cape Town, Cape Town, South Africa, 5Centre for Health Economics Research and Evaluation, Sydney, Australia, 6National University of Singapore, Singapore, Singapore.
1SCHARR, University of Sheffield, Sheffield, United Kingdom, 2ScHARR, University of Sheffield, Sheffield, United Kingdom, 3University of Otago, Dunedin, New Zealand, 4University of Cape Town, Cape Town, South Africa, 5Centre for Health Economics Research and Evaluation, Sydney, Australia, 6National University of Singapore, Singapore, Singapore.
OBJECTIVES: Content validity is a fundamental property of patient-reported outcome measures (PROMs). Although general guidance exists, no framework is tailored specifically to generic preference-weighted measures (PWMs), potentially leading to suboptimal study design and misleading conclusions about validity across populations. This scoping review aimed to identify and summarise qualitative methods used to assess the content validity of generic PWMs.
METHODS: The review followed Joanna Briggs Institute guidance. Searches of eight databases identified peer-reviewed studies reporting primary qualitative research on relevance, comprehensibility, and/or comprehensiveness of generic PWMs. Two reviewers independently screened studies and extracted data on study characteristics, participants, instruments, and methods. Study trustworthiness was evaluated using Lincoln and Guba’s framework (credibility, transferability, dependability, and confirmability).
RESULTS: Searches conducted in December 2025 and updated in April 2026 identified 46 eligible studies. Most were undertaken in a single country (42/46), commonly the UK (15/46). Nineteen PWMs were evaluated, most frequently the EQ-5D-5L (31 studies). Thirty-two studies were qualitative and 14 used mixed-methods. Evidence relating to relevance was reported in 37 studies, comprehensibility in 29, and comprehensiveness in 32. However, key instrument features were assessed less often, including completion instructions (6 studies), recall period (26), and response levels (30). Reporting was generally limited: 42 studies provided only illustrative quotations, three reported no qualitative data, and only one reported a complete dataset. Study trustworthiness varied across domains, with credibility scoring lowest (22%). Overall transparency was poor, with frequent omission of methodological details; only 14 studies reported using a reporting checklist.
CONCLUSIONS: Considerable variation exists in qualitative approaches to assessing the content validity of generic PWMs, with inconsistent alignment to good practice guidance. Important gaps remain in evaluating instrument design features, while limited reporting restricts assessment of study quality and trustworthiness. PWM-specific guidance is needed to improve the design, conduct, reporting, and interpretation of qualitative content validity research.
METHODS: The review followed Joanna Briggs Institute guidance. Searches of eight databases identified peer-reviewed studies reporting primary qualitative research on relevance, comprehensibility, and/or comprehensiveness of generic PWMs. Two reviewers independently screened studies and extracted data on study characteristics, participants, instruments, and methods. Study trustworthiness was evaluated using Lincoln and Guba’s framework (credibility, transferability, dependability, and confirmability).
RESULTS: Searches conducted in December 2025 and updated in April 2026 identified 46 eligible studies. Most were undertaken in a single country (42/46), commonly the UK (15/46). Nineteen PWMs were evaluated, most frequently the EQ-5D-5L (31 studies). Thirty-two studies were qualitative and 14 used mixed-methods. Evidence relating to relevance was reported in 37 studies, comprehensibility in 29, and comprehensiveness in 32. However, key instrument features were assessed less often, including completion instructions (6 studies), recall period (26), and response levels (30). Reporting was generally limited: 42 studies provided only illustrative quotations, three reported no qualitative data, and only one reported a complete dataset. Study trustworthiness varied across domains, with credibility scoring lowest (22%). Overall transparency was poor, with frequent omission of methodological details; only 14 studies reported using a reporting checklist.
CONCLUSIONS: Considerable variation exists in qualitative approaches to assessing the content validity of generic PWMs, with inconsistent alignment to good practice guidance. Important gaps remain in evaluating instrument design features, while limited reporting restricts assessment of study quality and trustworthiness. PWM-specific guidance is needed to improve the design, conduct, reporting, and interpretation of qualitative content validity research.
Conference/Value in Health Info
2026-11, ISPOR Europe 2026, Vienna, Austria
Value in Health, Volume 29, Issue 12S
Code
P12
Topic
Economic Evaluation, Patient-Centered Research, Study Approaches
Topic Subcategory
Instrument Development, Validation, & Translation, Patient-reported Outcomes & Quality of Life Outcomes
Disease
No Additional Disease & Conditions/Specialized Treatment Areas