OBJECTIVES: To perform a literature review of the psychometric methods used in validation studies of newly developed patient-reported outcomes measures (PROMs).

METHODS: All validation studies of PROMs, published in 2014 and 2015, were retrieved from the following online databases, according to a pre-specified search strategy: MEDLINE, PsycINFO, and EBM Reviews–Cochrane Database of Systematic Reviews. Eligibility criteria were established and applied by two investigators to include suitable studies. Data on psychometric measurement were extracted and reported according to the 3 main quality domains (validity, reliability, and responsiveness).

RESULTS: The search strategy identified a total of 5254 articles. Out of this pool, 158 articles reporting validation studies met the inclusion criteria. In total, most of the studies reported at least one analysis for validity (98%) or for reliability (94%). Validity (content validity, structural validity and at least one hypotheses testing analyses) was completely assessed in only 31% of the included studies. Construct validity (structural validity and hypotheses testing) was completely assessed in only 8% of the included studies. Particularly, concerning hypotheses testing, only 14% of the included studies had tested the 3 aspects of this propriety (i.e., convergent, discriminant and discriminative validity) and a great proportion did not formulate hypotheses regarding expected correlations in advance. Criterion validity was assessed in 23% of the studies. Concerning reliability analyses, 88% and 63% of the studies measured internal consistency and the test-retest reliability, respectively. Only 53% of the studies assessed these two properties. Inter-rater, intra-rater reliability or measurement error were tested in less than 2% of the studies. Finally, floor and ceiling effects were rarely mentioned (35%).

CONCLUSIONS: The results of this review highlight that the available evidence on the measurement properties is mostly limited. In fact, a lot of information regarding the measurement properties of the original version of PROMs is still lacking.