The following is taken from the Third Plenary
Session, “The Patient's Voice in Medical Product
Evaluation: FDA Draft Guidance on Measuring
PROs,” presented at ISPOR 11th Annual International
Meeting, May 2006, Philadelphia PA USA
Patient-reported outcomes (PROs) represent the
patient's voice in the drug and device development
process. As an outcome in a clinical trial, a PRO
operationalizes how patients survive, feel, or function,
from the perspective of the patients themselves.
In cases where treatment effects are
known only to the patient, direct, formal query of
patient perceptions is the only valid measurement
approach. Symptom frequency, severity, and
impact, for example, are obligatory PROs. In
cases where physiologic effects can be clinically
observed or quantified, such as tumor size or pulmonary
function, PROs quantify the extent to
which these specific effects translate into perceptible
and meaningful outcomes to patients. In
today's health care environment of informed consumers,
evidence-based practice, and concern
about resource allocation, data from PROs are an
essential element of the decision-making process.
As the importance of the patient's voice in the
drug and device development process is realized,
the need for valid instruments to evaluate these
outcomes in clinical trials has increased.
Developing a new scientific instrument for quantifying
health-related phenomena, at the bench or
at the bedside, isn't easy. It requires an investment
of time and resources in order to clearly
define the phenomena, build the instrument, subject
the instrument to tests of reliability and validity,
and then observe its use under varied situations
to fully understand and utilize the metric or
unit of measurement it produces to represent the
phenomena of interest. As the metric takes hold
in science and potentially, although not necessarily,
in clinical practice, the use of the instrument
and the metric increases, and the interpretation
of the metric in terms of “high” and “low” scores
and the magnitude of within-group change and
between-group differences is enhanced.
Instruments to measure body temperature, blood
pressure, expiratory flow rates, and other clinical
indicators have followed this path - from the laboratory
to clinics and ultimately, to patient homes
where many are used as part of health monitoring
and self-care regimens.
Like their physiologic endpoint counterparts,
PRO instruments are becoming more widely
used in clinical research, and with this use
comes a greater understanding of their associated
metrics to summarize given health states or
treatment effects. The SF-36, a generic instrument
with a relatively long history [1-3], is an
example of a PRO instrument that is widely used
with a known metric whose scores are becoming
easier to interpret as it is used over time.
Unfortunately, the generic nature of the instrument
can make it less sensitive to treatment
effects and, in some cases, actually inappropriate
for evaluating treatment efficacy due to Type
II error. Measuring outcomes specific to a given
disease or treatment requires instruments that
capture the unique features of the outcome.
These instruments are grounded in patient
descriptions of the problem within the context of
the underlying disease or health state and the
understanding that the problem can be allayed or
eliminated with treatment.
Clearly, evaluating the effect of treatment on
PROs is an important part of the drug and device
development process. Unlike traditional physiologic
efficacy endpoints, however, standard PRO
measures are rarely used. An example of a possible
exception is the Health Assessment
Questionnaire (HAQ) [4, 5], an instrument commonly
used to evaluate physical function/disability
outcomes in clinical trials of rheumatoid
arthritis. Used in clinical research generally and
pharmaceutical trials specifically, information on
the effect of treatment on HAQ scores has
appeared in ten pharmaceutical labels over the
past seven years . More commonly, a given PRO
is assessed using any one of a number of instruments.
Although each instrument may be valid
and reliable for this purpose, differences in structure
and scoring make cross study comparison
and the development of interpretive “intuition”
more difficult.
From both scientific and efficiency perspectives,
the idea of a set of standard PRO instruments
acceptable to regulatory authorities and used by
multiple pharmaceutical or device companies in
the development process has great appeal.
Rather than each company developing its own
instrument for measuring specific clinically-relevant
outcomes, it would be more cost-effective if companies would join forces to develop a single,
standard instrument that can be used in their
respective trials, and in the broader field of health
care research as well. Like physiologic efficacy
endpoints, the new instrument(s) would be used
as the standard for assessing the specified PRO
in clinical trials, enabling companies, regulatory
authorities, clinical research experts, and decision-
makers to understand and orient to a single
metric for quantifying a given PRO.
This is the intent of the EXACT-PRO initiative, the
first attempt to bring together experts in instrument
development and validation, specialists in
clinical practice and research, and dedicated
staff from the U.S. Food and Drug Administration
(FDA) to develop a single, validated, and accepted
patient-reported outcome (PRO) instrument
for use in drug development trials. A key element
of the project has been the interest, enthusiasm,
and sponsorship of pharmaceutical companies
committed to improving PRO evaluation in chronic
pulmonary disease through this innovative
cooperative, collaborative approach.
The EXACT (EXacerbations of Chronic pulmonary
disease Tool) project involves the development of
an instrument for evaluating the frequency,
severity and duration of acute exacerbations of
chronic obstructive pulmonary disease (COPD)
in general, and chronic bronchitis specifically.
Exacerbations are an important feature of COPD,
leading to significant morbidity and mortality.
Despite widespread interest in understanding the
effect of treatment on exacerbations, there is no
consensus on its definition or evaluation. A variety
of outcome measures, often with areas of
overlapping content and/or different structure,
have been used in clinical trials, with no single,
validated, accepted “gold” standard. This has
created problems for the industry, regulatory
agencies, and clinical decision makers as they
attempt to evaluate the absolute and relative efficacy
of new treatments for this important health
problem. The EXACT-PRO initiative brings
experts together to address these problems.
The EXACT will be structured to indicate the
presence of an acute exacerbation in order to
evaluate exacerbation frequency as an outcome
in maintenance therapy or exacerbation prevention
trials of COPD. In addition, it will be designed to evaluate the effect of treatment in terms of
severity, duration and resolution of exacerbations
from the perspective of the patients themselves.
The result of the project will be a single instrument
sponsors can use as an efficacy endpoint
in clinical trials of investigational drugs to be submitted
to the FDA for review and approval and to
communicate patient-reported outcomes of
treatment to decision makers.
The project is being conducted in two phases:
Phase I, completed in the fall of 2006, included a
literature review, summary, and analysis; patient
focus groups and one-on-one interviews; two
expert panel meetings with experts from clinical
research and the FDA to address the patient's
perspective of exacerbations, the item pool and
response options; draft questionnaire development;
cognitive debriefing interviews; and the
development of a draft validation protocol. Phase
II of the project is now underway and includes
finalizing the measure, the final round of cognitive
debriefing interviews, implementation of the validation
protocol, data analysis, and a third expert
panel meeting to discuss the final instrument and
its empirical performance properties.
To make certain sponsors, experts, and the FDA
have access to comprehensive, timely information
from and about the project, the EXACT-PRO
initiative includes a limited access web site
(www.exactproinitiative.com). Log-in passwords
give sponsors, experts, and the FDA access to
the project's comprehensive bibliography on
acute exacerbations of chronic bronchitis and
COPD, instrument analysis tables, study protocols
and interview guides, and updates on study
progress and results. A dialogue page serves as
a medium for asking questions and sharing ideas
about the PRO instrument development process.
PROs represent the patient's voice in the drug
and device evaluation process. It is important
this voice is heard. One approach to improving
the science and strengthening the patient's voice
is to develop standard PRO instruments to
assess specified outcomes in selected disease
areas. This requires cooperation and collaboration
of experts from measurement, the clinical
research field, regulatory agencies, and industry.
Are we ready for the challenge?
Acknowledgements
UBC's EXACT-PRO team includes Nancy Kline
Leidy PhD, Director and Principal Investigator of
the project, Terry Wilcox PhD, Kellee Howard,
Jennifer Petrillo, Charlotte Cates, Elizabeth Allan,
and Alise Nacson, with support from M.A.
O'Donnell, Julia Dixon, Laurie Smith and other
members of the UBC staff. Drs. Paul Jones and Sanjay Sethi are serving as Senior Clinical
Research Consultants. The EXACT-PRO expert
panelists include experts in PROs, pulmonary
research, and members of the FDA.
Phase I of the EXACT-PRO Initiative was possible
through the commitment of the following sponsors:
Adams Respiratory, AstraZeneca, Bayer,
Boehringer-Ingelheim, Forest Laboratories,
Merck, Novartis, Ortho-McNeil, Pfizer, and
Schering-Plough.
REFERENCES
- Ware JE Jr, Sherbourne CD. The MOS 36-item short-form health
survey (SF-36). I. Conceptual framework and item selection. Med
Care 1992;30:473-83.
- McHorney CA, Ware JE Jr, Raczek AE. The MOS 36-Item Short-
Form Health Survey (SF-36): II. Psychometric and clinical tests of
validity in measuring physical and mental health constructs. Med
Care 1993;31:247-63.
- McHorney CA, Ware JE Jr, Lu JF, Sherbourne CD. The MOS 36-
item Short-Form Health Survey (SF-36): III. Tests of data quality,
scaling assumptions, and reliability across diverse patient groups.
Med Care 1994;32:40-66.
- Ramey D, Fries J, Singh G. The Health Assessment Questionnaire
1995: Status and review. In: Spilker B, ed. Quality of Life and
Pharmacoeconomics in Clinical Trials (2nd ed.). Philadelphia:
Lippincott-Raven; 1996.
- Bruce B, Fries J. The Stanford Health Assessment Questionnaire
(HAQ): A review of its history, issues, progress and documentation.
J Rheumatol 2003;30:167-78.
- Source: UBC PRO Database, described in “UBC's database on
patient-reported outcomes in approved pharmaceutical labeling.”
UBC ResearchNews 2006;12:18-19.
