Good Research Practices for Establishing and Reporting Evidence of the Content Validity of Patient-Reported Outcomes Instruments

Chair:
Donald L. Patrick PhD, MSPH, Professor, Department of Health Services, University of Washington, Seattle, WA, USA

Leadership Team:
Laurie B. Burke RPh, MPH, Director, Study Endpoints and Label Development, Office of New Drugs, CDER, FDA, Washington, DC, USA
Chad Gwaltney PhD, Assistant Professor (Research), Department of Community Health, Brown University, Providence, RI, USA and Senior Scientist, PRO Consulting, Pittsburgh, PA, USA
Nancy Kline Leidy PhD, Senior Vice President of Scientific Affairs, United BioSource, Bethesda, MD, USA
Mona L. Martin RN, MPA, Executive Director, Health Research Associates, Inc, Seattle, WA, USA
Lena Ring PhD, Principal Scientist in PRO Strategy, HEOR, R&D, AstraZeneca, Södertälje, Sweden and Associate Professor, Pharmaceutical Outcomes Research, Department of Pharmacy, Uppsala University, Sweden

Goal:
The task force will develop the following topics and provide recommendations for the following research methods:

• Methods for generating items for a new PRO instrument including item wording, evaluating the completeness of item coverage, and performing initial assessments of clarity and readability.  These are often called concept\item elicitation studies.
• Methods for documenting item development including the qualitative interview guide used in item elicitation, sample composition (inclusion and exclusion criteria) and size, training of interviewers, and methods for demonstrating and documenting that saturation has been achieved or no new content is being elicited.  Recommendations will be developed for each of these subtopics.
• Methods for coding of qualitative data from item generation, development of the qualitative coding frame, and application of this frame including evaluation of inter-coder reliability.
• Methods of cognitive interviewing including patient understanding of items and  evaluating appropriateness of recall period and response options
• Methods for tracking item development through the various stages of research and preparing this tracking for submission to regulatory agencies.

Background / Overview:
Content validity is the extent to which an assessment instrument measures the important aspects of the concept assessed.   A patient reported outcome (PRO) instrument measures a concept relevant and important to the patient’s condition and its treatment.  For PRO instruments, items and domains as reflected in the scores of an instrument should be important to the target population of patients and comprehensive with respect to patient concerns concerning the concept being assessed.  Documentation of target patient population input in item generation, as well as evaluation of patient understanding through cognitive interviewing, can provide the evidence for content validity.

The importance of PRO instrument content validity is stressed both by the US Food and Drug Administration Draft Guidance (US Food and Drug Administration, 2006) and European Medicines Agency Reflection Paper on the Regulatory Guidance for the Use of Health Related Quality of Life (HRQL) Measures (EMEA, 2006).  These guidance documents describe how PRO or HRQL measures are used as effectiveness end points and how study results are used to support approval or in the US context, label claims and advertising.  Although these documents include discussions on instrument evaluation, study design, and data analysis, they do not provide developers or industry specific recommendations on how to gather and prepare documentation of content validity.  The FDA guidance provides a summary of the kinds of evidence needed to support a conclusion that content validity is adequate for the purpose proposed.

A previous ISPOR Good Practices Task Force has taken up the issues involved in the evaluation of content validity of existing PRO instruments, Evaluating and Documenting the Content Validity of Patient-Reported Outcome Instruments to Support Medical Product Labeling and Promotional Claims in the United States: Report of the ISPOR PRO Existing Instruments & Their Modifications Task Force.(Rothman, 2009).  http://www.ispor.org/TaskForces/documents/Rothman.pdf   The report outlines the broad need for documentation and discusses the issues involved when existing PRO instruments are modified or evaluated for content validity.

However, it does not take up the specific methodological practices involved in (1) designing studies to gather evidence of content validity and (2) methods for evaluating and documenting content validity.  This proposed TF report will build upon this work with a more specific focus on two methodological issues within the scope of a manuscript submitted for review by ISPOR membership and submitted for publication to Value in Health.

Timeline effective 3/31/10:

Task Forces Index

Task Force Members Access Only

  How to login

User name: first part of the email address (before @)
e.g. for jdoe@ispor.org it should be jdoe)
Password: ISPOR Member ID.
4
   Click here for login help