WW19 METHODOLOGICAL ISSUES IN MEASURING AND INFLUENCING ANTIBIOTIC PRESCRIBING IN
HOSPITALS
Davey PG1, Ofori BD2.
1University of Dundee, Dundee, Scotland, United Kingdom; 2Ninewells Hospital and
Medical School, Dundee, Scotland, United Kingdom
Learning Objectives: On both sides of the Atlantic hospitals have been charged
with ensuring the prudent use of antibiotics to minimise the emergence and
spread of antibiotic resistance. An ongoing systematic review of interventions
to change antibiotic prescribing has shown that 70% of published papers have
simple but fundamental methodological flaws. This workshop will provide examples
of good and bad practice and review some web-based resources for methodological
advice and support.
Who Would Benefit: Researchers who want to measure the outcomes of drug
treatment or the impact of interventions to change prescribing. Stakeholders
involved in Hospital Formulary selection or review. Decision makers who
critically appraise quality assurance programs and performance measures for
hospital antibiotic prescribing. Editors and referees for journals that publish
work on interventions to change prescribing.
A systematic review of “Interventions to improve antibiotic prescribing
practices for hospital inpatients” (the full protocol is available on the
Cochrane Database) has revealed that 70% of current publications do not meet the
criteria set by the Effective Practice and Organisation of Care group. We will
review the two commonest errors: 1) use of uncontrolled before and after design
and 2) flawed design of interrupted time series. Flawed analysis of interrupted
time series is less serious because it can be corrected provided that adequate
data are presented in the paper. Hospitals are under pressure to demonstrate
improvements in prescribing and to show that thy are controlling antibiotic
resistance. Unfortunately most hospitals only have access to information about
prescribing at the ward level rather than at individual patient level. We will
review evidence for ecological bias in studies using ward level data. We will
provide participants with information about ESAC (European Surveillance of
Antimicrobial Consumption), an EU funded project designed to harmonise data
collection and analysis across EU member states (http//esac.www.uia.ac.be/esac/)
WW20 METHODOLOGICAL CHALLENGES IN RETROSPECTIVE DATABASE STUDIES: STEPS FOR GETTING
IT RIGHT
Menzin J, Lang K
Boston Health Economics, Waltham, MA, USA
Learning Objectives: The objectives of this workshop are to describe key
methodological challenges that affect the validity of retrospective database
studies, to offer possible solutions, and to highlight issues that have not yet
been resolved by outcomes researchers.
Who Would Benefit: Persons involved in the design, analysis, or management of
retrospective database studies would benefit from this workshop.
Administrative claims are increasingly used for outcomes research studies,
despite important methodological challenges. These challenges include, among
others, assessing the validity of using diagnostic and procedural codes for case
identification and outcomes assessment, controlling for various forms of
confounding, determining the adequacy of sample size, and addressing the
representativeness of the health plan and patient population examined. Most of
these problems directly influence the internal and/or external validity of a
study. The impact of many of these issues can be assessed in varying degrees
using validation substudies (e.g. to assess the accuracy of diagnostic coding),
appropriate statistical techniques (e.g. matching, regression analysis,
propensity scores), and sensitivity analyses (e.g. to assess the impact of
alternative definitions of cases or outcomes). Despite these assessment tools,
it often is not easy to know how well these problems actually have been resolved
in any given study. In this workshop, the various challenges that may impact the
validity of retrospective database studies will be discussed, and potential
methods for addressing such problems will be described. Published examples and
case study insights will be used in presenting possible solutions. The extent to
which each of the relevant hurdles can be addressed adequately through novel
supplemental data or innovative statistical techniques will be highlighted, and
an appraisal also will be made of the problems and challenges that outcomes
researchers have not yet fully resolved.
WW21 ISSUES IN THE SPECIFICATION OF PERFORMANCE MEASURES: INSIGHTS FROM THE STUDY OF
CLINICALLY RELEVANT INDICATORS OF PHARMACOLOGIC THERAPY (THE SCRIPT PROJECT)
Edward W, Kogut S
Massachusetts Peer Review Organization, Inc, Waltham, MA, USA
Learning Objectives: To learn of issues in constructing indicators of quality of
medication use for the outpatient setting; including the effect on performance
score for measures of varying specification, and implications for agreement upon
a standardized measure set.
Who Would Benefit: Those interested in the measurement of quality of medication
use as a component of optimizing drug utilization.
Performance measurement can be used to assess and improve the quality of
medication use in populations. The utility of performance measurement can be
enhanced when agreement exists on measure specification. To examine the effect
on performance scores when varying measure specifications. The SCRIPT Project
was the first initiative of the Coalition for Quality in Medication Use, and was
funded by the Centers for Medicare and Medicaid Services. The goal of the SCRIPT
project was to develop valid and reliable measures of quality in medication use
to be used in the outpatient setting. The SCRIPT performance measures were
tested using data provided by health plan and medical group practice
collaborators from eight states. Measures were constructed using various
specifications, and the effect on performance (score) was determined. For most
measures, modifying specifications had a small or no effect on the performance
score. Excluding patients having contraindications to therapy or other potential
decision modifying factors from the measure had little impact on scores for the
use of angiotensin-converting enzyme inhibitor (ACEI) use in heart failure,
beta-blockers in post-myocardial infarction, and lipid-lowering therapy in
patients with dyslipidemia. In a large national study, differently specified
measures generated similar performance scores in most instances. Disagreement
over measure specification should not impede the utilization of performance
measures for quality improvement.
Fehnel SE, McLeod LD
RTI Health Solutions, Research Triangle Park, NC, USA
Learning Objectives: This workshop is intended to be an introduction to
psychometrics and its value in outcomes research. Our goal is to equip
participants with a basic understanding of the most commonly used psychometric
techniques and criteria appropriate for evaluating the utility of outcomes
instruments.
Who Would Benefit: Anyone who wants to understand or be a better consumer of
psychometric information could benefit by attending.
This workshop is intended to be an introduction to psychometrics and its value
in outcomes research. The purpose and general principles behind the most
commonly used psychometric techniques such as exploratory factor analysis and
structural equations modeling, as well as various approaches to item response
theory (e.g. Rasch, 2-parameter logistic and graded-response models) will be
described. In addition, for each psychometric property such as reliability,
validity, and responsiveness, a bit of history, definitions, estimation methods,
and criteria appropriate for evaluating the utility of outcomes instruments will
be provided. Real-life examples will be used to illustrate what tradeoffs among
various instrument attributes may be appropriate given specific research goals.
An interactive exercise will be included to give the audience practice in
comparing instruments with different psychometric strengths and weaknesses.
During the remaining minutes of the workshop, the presenters will entertain any
questions the audience may have about psychometrics or the selection of outcomes
instruments. Additional issues that may be considered include when it is
imperative to use a fully validated instrument as opposed to situations in which
a new instrument or set of survey items may be reasonable.
WW23 PAVING A ROAD TO SUCCESS: HOW TO OBTAIN AN OUTCOMES RESEARCH FELLOWSHIP WITHIN
THE PHARMACEUTICAL INDUSTRY
Carter CT, Maio V, Pizzi L, Lofland J
Thomas Jefferson University, Philadelphia, PA, USA
Learning Objectives: Pharmaceutical outcomes research fellowships provide a
combination of on-the-job experience and didactic training required to secure a
position within the pharmaceutical industry. This workshop will educate
interested candidates about fellowship opportunities and provide a framework to
critically appraise and select a fellowship that matches their interest and
experience level.
Who Would Benefit: Students who are interested in pursuing an industry-sponsored
health outcomes or pharmacoeconomic fellowship, as well as sponsors and
preceptors of fellowships.
Successful pharmaceutical outcomes research endeavors require an understanding
of research design, epidemiology, health policy, health economics, and drugs.
The objective of this workshop is to educate candidates about industry-sponsored
outcomes research fellowships, and to work through the factors that candidates
should consider when evaluating fellowship programs. Participants will be
informed of opportunities that are available, as well as provided an overview of
the application process. Factors that sponsors and preceptors consider when
evaluating candidates will be described in detail. Findings from a recently
completed ISPOR-sponsored survey of pharmacoeconomic fellowship programs will be
presented. Then, through an interactive session, participants will identify and
assign utility values to educational, personal, and professional attributes of a
sample fellowship program. With over eight years of experience in serving as a
first-year fellowship site, the Office of Health Policy and Clinical Outcomes at
Thomas Jefferson University developed this decision-making methodology as a tool
for prospective fellows. Workshop participants will be able to use this tool to
assist them in their pursuit of an industry-sponsored outcomes research
fellowship.
WW24 RISE OF CONSUMERISM – UNDERSTANDING CONSUMER SOVEREIGNTY
Heissel A
Ethicon Endo-Surgery (Europe) GmbH, Norderstedt, Germany.
Learning Objectives: The concept of consumer sovereignty often enters into the
discussion of health policy. However, there is often a lack of clarity of the
concepts behind the catchword. The objective of this workshop is to clarify the
underlying theoretical concepts of consumerism. On the one hand is the gain of
trade approach opposed on the other hand by the allocational approach. Both
approaches have different underlying human behaviors and come, therefore, to
different conclusions regarding the applicability of consumerism.
Who Would Benefit: Everybody who is confronted with the term in one way or the
other will benefit from a clarification of the concepts of consumerism and the
underlying human behaviors. Understanding the different notions of the term
might help to avoid miscommunications and to understand how consumerism might
influence outcomes research.
The first part of this workshop will summarize the theoretical concepts of
consumerism and will scrutinize the concepts of human behavior. This will be
followed by an interactive discussion of the models of human behavior and their
value for real life problems. The second part of the workshop will focus on an
explanation why, even in market-driven economies, consumerism is a relatively
new concept with respect to health care systems and why very often consumers are
seen to be not able to express their own needs. The interactive discussion of
this part of the workshop will focus on the opportunities and limits which arise
from the concept of consumerism in the health care environment with respect to
better health and financing issues. In a guided discussion, the workshop
participants will brainstorm on the question how different notions of
consumerism might affect the work of outcomes researchers.
WW25 MEASURING VALUE IN THE DRUG DEVELOPMENT PROCESS: HOW DO YOU KNOW IT IS THERE? Paul JE1, Backhouse ME2.
1RTI Health Solutions, Research Triangle Park, NC, USA; 2RTI Health Solutions,
Manchester, United Kingdom
Learning Objectives: The objective of this workshop is to provide clarity around
the concept of value in the drug development process through presentation and
discussion of a model that delineates various dimensions that contribute to the
overall drug value proposition.
Who Would Benefit: Persons who are involved in “value-added” activities in the
pharmaceutical or medical device development process, either as decision-makers
or implementers.
“Value” is a term that is often used in the drug development process without a
clear understanding or mutual agreement as to its meaning. Payers, providers,
patients, and manufacturers all define value differently. The overall value
proposition for a pharmaceutical product needs to be specified early in the
development process and take into account (and weight as appropriate) the value
attributes important to the various perspectives you wish to target. “Managing
to Value” offers a paradigm for tracking the components of value over time,
assessing whether or not sufficient value still exists at different decision
points, and using this information to inform decisions regarding product
development. This workshop will present a model of value as being comprised of
time (time efficiency), investment (cost efficiency), and quality (product
characteristic) dimensions. The balance among these dimensions determines
product value. The model is further extended to show how these dimensions impact
the product cash flow curve over the course of full development and
commercialization. The interactive portion of the workshop will involve
specification by and debate among the participants with regard to the value
dimensions and appropriate measures. Prototype examples of products (e.g., a
potential blockbuster, “me-too/me-better” follow-ons, etc.) will be worked
through within the “Managing to Value” model.
WW26 MAXIMIZING THE SCIENTIFIC AND STRATEGIC VALUE OF PATIENT REGISTRIES
Trotter J, Larson L, Vreeland MG
Ovation Research Group, Highland Park, IL, USA
Learning Objectives: This workshop will establish patient registries’ ability to
capture real-world clinical, economic, and humanistic data, and to demonstrate
the need for the health economics community to play a leading role in these
research initiatives. The workshop will establish the scientific and strategic
goals underlying patient registries, examine the role of registries relative to
other pharmacoeconomic approaches, and present an approach for their development
and implementation.
Who Would Benefit: Health economists seeking to better understand patient
registries and their advantages relative to other outcomes research initiatives.
The objectives underlying industry-sponsored patient registries — typically
large, observational, prospective research programs involving data collected
from physicians and, often, directly from patients — vary from safety
surveillance to market penetration. Accordingly, these programs are designed and
implemented by staff with a broad range of responsibilities, from marketing and
product management to clinical research. The most successful patient registries
achieve strategic value for the sponsor while serving as a vehicle for
scientific exploration into the clinical, economic, and humanistic impact of a
drug, device, or disease state. As such, outcomes researchers should — but
seldom do — play a central role in the development of registries. This workshop
will begin with an interactive discussion focusing on the audience’s knowledge
of and experience with patient registries. The workshop will then present brief
case studies of several existing registry programs and their respective
histories, achievements, and successes. A four-step approach for designing and
executing patient registries will be presented, including techniques employing
state-of-the-art technologies for data collection and communications. The
workshop will conclude with suggestions for bringing coordination of patient
registries under the authority of health economics and outcomes research
departments.
WW27 EVALUATION OF HEALTH ECONOMIC INFORMATION USED IN PROMOTION
Piault E, Cronin K, Mathieu J, Hu EJ, Burke LB.
Food and Drug Administration, Rockville, MD, USA
Learning Objectives: A qualitative review of industry-disseminated economic
models according to a set of criteria derived from the Food and Drug
Administration Modernization Act (FDAMA) legislative history and the health
economic literature.
Who Would Benefit: Health Care provider members involved with health care
decision-making.
The Federal Food, Drug and Cosmetic Act (FFDCA §502(a)) as amended by the Food
and Drug Administration Modernization Act (FDAMA §114) provides a unique
standard, competent and reliable scientific evidence, for FDA’s evaluation of
economic information. The economic information must also be consistent with the
product’s approved indications. In addition, FDA recognizes that industry
responds to specific unsolicited requests for economic information and, in some
cases, prepares such responses according to standard formats. We performed a
qualitative review of industry-disseminated economic models according to a set
of criteria derived from the FDAMA legislative history and the health economic
literature. The review included information intended for both managed care and
other audiences and included responses to unsolicited requests. The primary
focus of the review was to determine the extent to which the models were
directly related to approved labeling and based on competent and reliable
scientific evidence. The review revealed a wide range of quality, accuracy, and
transparency in data used to develop the models. In nearly all cases, the models
made assumptions about treatment outcomes that were not directly related to the
approved indications for the product. In addition, most models failed at least
one of the economic criteria. Industry-disseminated economic models are of
variable utility in health care decision-making. The differences between those
models used in promotion and those included in responses to inquiries will be
presented. The regulatory implications of these findings will be discussed.
