Characterization of Variability in Response Set Composition Across Pros: Insights for Reducing Complexity and Improving the Patient Experience of Clinical Research

OBJECTIVES: Ordinal scales with discrete response options commonly appear in patient reported outcomes (PROs) to assess various dimensions (e.g., frequency, severity) of health conditions or treatments. Previous research demonstrates impact on patient comprehension related to increased response set size and poor conceptual spacing between options. Other potential problems for patients are within and across instrument variability in the composition of response sets assessing a dimension, and response set balance, defined as equivalent numbers of response options describing high and low target dimension values. We aimed to characterize these potential problems by examining the range of variability in response set composition and balance in existing PROs, specifically for the frequency dimension.

METHODS: We reviewed ~2000 PROs from previous linguistic validation projects, extracting 47 unique ordinal frequency response sets, each containing 3 to 7 response options (3 options: 13%; 4:15%; 5: 57%; 6: 11%; 7: 4%).

RESULTS: Across all 47 sets, 74% had an odd number of options and thus a natural midpoint. 13 unique response options represented the lowest frequency (“initial position”), 15 represented the highest frequency (“final position”), and 11 represented set mid-points. “Never” was the most frequent initial position option, appearing there 47% of the time; “always” was the most frequent final position option at 26%; “sometimes” was the most frequent mid-point at 51%. 22 unique options appeared outside of initial, final, and middle positions. Qualitative analysis by instrument design experts found 28% of frequency sets were unbalanced (e.g., “sometimes, most of the time, all the time”).

CONCLUSIONS: Results show great variability in frequency response set composition and the identity of initial, final and mid-point anchors, and a high percentage of unbalanced sets. These factors may reduce response set comprehensibility and impact PRO function and data comparability within and across trials, demonstrating a need for increased homogeneity of response sets assessing particular dimensions.