GRP logo
Evidence Needed to Support Measurement Equivalence between Electronic and Paper-Based PRO Measures

Recommendations on Evidence Needed to Support Measurement Equivalence between Electronic and Paper-Based Patient-Reported Outcome (PRO) Measures: ISPOR ePRO Good Research Practices Task Force Report.

The citation for this report is:
Coons SJ, Gwaltney CJ, Hays RD, et al. Recommendations On Evidence Needed To Support Measurement Equivalence Between Electronic And Paper-Based Patient-Reported Outcome (PRO) Measures: ISPOR ePRO Good Research Practices Task Force Report. Value Health 2009:12:419-429.

Task Force Chair:
Stephen Joel Coons, PhD, Professor of Pharmacy and Public Health, University of Arizona, Tucson, AZ

Leadership Team:
Ethan Basch, MD, MSc, Health Outcomes Research Group, Departments of Biostatistics and Medicine Memorial Sloan Kettering Cancer Center, New York, NY
Laurie B. Burke, RPh, MPH, Director, Study Endpoints and Label Development Office of New Drugs, CDER, FDA Washington, DC
Don Bushnell, MA, Associate Director, Health Research Associates, Seattle, WA
David Cella, PhD, Professor, MED-Psych & Behavioral Science, Northwestern University, Evanston, IL
Paula Funk Orsini, PhD, Principal, Funk & Orsini Associates Scarsdale, NY
Chad J. Gwaltney, PhD, Center for Alcohol and Addiction Studies, Brown University, Providence, RI and ePRO Consultant, Invivodata
Ron D. Hays, PhD, Professor, UCLA Department of Medicine, Division of General Internal Medicine & Health Services Research, Los Angeles, CA
William Lenderking, PhD, Director/Team Leader, Outcomes Research Group, Pfizer Inc., Groton, CT
Dennis Revicki, PhD, Director, Center for Health Outcomes Research The MEDTAP Institute at UBC Bethesda, MD
James W. Shaw, PhD, PharmD, MPH, Assistant Professor, Department of Pharmacy Administration, University of Illinois at Chicago, Chicago, IL
Saul Shiffman, PhD, Senior Scientific Advisor, Pinney Associates, Bethesda, MD and Professor, Psychology & Pharmaceutical Science, University of Pittsburgh, Pittsburgh, PA
Jeff Sloan, PhD, Professor, Division of Biostatistics, Mayo Clinic College of Medicine, Rochester, MN
Theron Taber, BS, President, Assist Technologies Scottsdale, AZ
Brian Tiplady, PhD, Senior Clinical Scientist, Invivodata, Edinburgh, Scotland
Keith Wenzel, BS, ePRO Principal, ClinPhone, Oregon, WI
Arthur Zbrozek, MSc, MBA, Senior Director, Wyeth Research, Philadelphia, PA

Concurrent with the increased use and significance of PROs in clinical trials has been the steady growth in electronic data capture (EDC) or eSourcing—the electronic collection of clinical trial data at the source (e.g., case report forms and PROs). EDC refers to the use of electronic systems for capturing clinical data in trials, and usually refers to systems where data are entered by research staff. ePRO refers to systems where electronic devices (e.g., computers) are used to capture data directly from the patient.

In February 2006, the U.S. Food and Drug Administration issued a draft version of Guidance for Industry titled “Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims.”

Draft FDA Guidance for Industry - Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims Draft Guidance

In Chapter IV: Evaluating PRO Instruments, Part D: Modification of an Existing Instrument, Section 4 addresses Changed Mode of Administration:

An instrument’s data collection mode is altered.  For example:

  • An interviewer-administered or supervised questionnaire is modified for self-administration (skip patterns can be a problem in this situation)
  • Paper-and pencil self-administered PRO is modified to be administered by computer or other electronic device (e.g., computer adaptive testing, interactive voice response systems, Web-based questionnaire administration, computer)
  • Instructions or procedures for administration within a trial differ from those used in validation studies (can alter the meaning of the responses from that of the original version)

The development of new PRO measures, specifically for EDC platforms, and the migration of existing paper-based PRO measures to electronic formats, present unique opportunities and challenges in outcomes research. 

Adaptation of clinical trial documents to eSource formats, including ePROs, must be accomplished in ways that assure the equivalence of data collected via paper and electronic methods.  Many PRO measures were originally developed to be administered via paper-based questionnaires; however, as with EDC in general, there is growing recognition of the many advantages of ePROs, including less administrative and subject burden, avoidance of secondary data entry errors, easier implementation of skip patterns, and more accurate and complete data.

The purpose of this task force is to set forth recommendations as to the evidence necessary to support the comparability or measurement equivalence of ePROs to the original paper-based PRO questionnaire from which they were adapted.

Migrating from paper-based modes of administration to electronic data collection devices is one of the most significant movements in the PRO measurement field.  Hence, in addressing the issue of changing modes of administration for PRO measures, ePRO will be the exemplar used by the task force.  Nevertheless, the good research practices developed are intended to be relevant to other administration mode changes.

Draft Final Report
The task force membership reflected a broad array of backgrounds, perspectives, and expertise that enriched the development of this report. A presentation and discussion period was held at the 12th Annual International ISPOR meeting and at the 11th Annual European Congress in Athens. In addition, a draft version of the full report was distributed to the 220 members of the ePRO reviewer group for comments. This report reflects the extensive internal and external input received during the 16-month good research practices development process. To view the Athens’ presentation:

ISPOR Good Practices For Outcomes Research Index