|
OUTCOMES ASSESSMENT
Multi-National Assessment of
Outcomes Via Retrospective Databases
Vittorio Maio PharmD, MS, MSPH, Elaine J. Yuen, Kenneth D. Smith; Jefferson Medical
College, Philadelphia, PA, USA; Diana I. Brixner, Gary M. Oderda, and Carl V. Asche,
University of Utah, Salt Lake City, UT, USA; Steve Morgan, University of British
Columbia, Vancouver, NC, Canada
Introduction
With the increased use of technology, data are
becoming the most mined and useful commodity
in any field, including health care. In recent years,
there has been an enormous growth of populationbased
health care databases, which represent an
important resource for clinical epidemiologists
and health care managers to answer a wide spectrum
of questions. These databases are usually
made up of documented hospitalizations, prescriptions,
and professional services. Over the last
decade health care claims databases have
expanded to include lab tests, value, vital signs
and other critical information.
With large sample sizes, at relatively low cost,
these databases have a great potential for research
applications. Population-based databases may be
used to study “real world” effectiveness and utilization
patterns of health services; to investigate
utilization and outcomes of therapeutics; to detect
adverse drug events; and to assess the impact of
health care policy implementation.
Despite these important benefits, the extent to
which individual countries have access to these
data is not readily known. In addition, the temptation
to utilize the combined information in these
databases for outcomes research on healthcarerelated
issues of global significance is strong;
however, there are few examples of such work.
This article will present health care databases from
the United States, Canada, and Italy, describing the
characteristics of these databases as well as their
similarities and differences. Operational, methodological,
and analytical challenges to conduct
multi-national retrospective outcomes research
evaluations will then be reviewed.
U.S. Data
An up and coming source for outcomes research
in the U.S. is the Electronic Medical Record (EMR)
database, GE Centricity, which is developed and
sold by General Electric Clinical Data Systems.
There are about 4,600 providers using this EMR
product in 35 states, which is growing rapidly at a
rate of roughly 20% per year. Physicians who use
the medical record system decide whether they
want their patients’ data to be available for
research purposes. The Medical Quality Improvement Consortium is the group of physicians
that allow their data to be used for research.
The Consortium’s primary purpose is to improve
patient care and to be involved in quality care management
and practice profiling. There is data on about 3.7 million patients in the
research database, which is about a quarter of the
patients whose providers use the EMR. There are a
large number of patients with most diseases. For
instance, there are approximately 421,000 patients
with hypertension and 623,000 with hyperlipidemia.
The data are sent nightly to Portland, OR,
USA where it is placed into a format that can be
used electronically for research purposes. Regarding characteristics of this database, the
data you would get is very similar to what you
would see going through a paper medical record.
Physician’s notes are not included due to the 1996
Health Insurance Portability and Accountability Act
(HIPAA) concerns that personal health information
may be included in the notes. There are advantages and disadvantages to getting
information from an EMR. In this particular case,
the first advantage is that the database is quite up
to date. Data from physicians’ offices are downloaded
on a daily basis to a central computer.
Another major advantage is that lab data are available,
a fact that distinguishes this database from
commercial insurance claims databases available
in the United States. There is clinical data available
related to diagnosis. Data on all patients are
included, regardless of whether the payor is government,
commercial insurance, or private.
Patients tend to stay with their physicians longer
than with an individual commercial insurance plan,
which lends to increased longitudinal data. The
median time that a patient is in the database is
three years. The database includes a record of all
vital signs. When a patient goes into a physician’s
office and has his/her vital signs taken, this information
is included as part of the research database.
All collected data is cleaned by an informatics
team and extracted for research purposes. As with any retrospective research database, there
are also limitations. A system like this is quite
expensive, both in terms of creating the software
and the technology infrastructure. In addition, conversion
of the data to a research oriented dataset is
difficult. The product was originally developed for
use in physician offices to help them with the
administrative aspects of their practice. There are
no patient identifiers, as required by HIPAA regulations,
and there is also no way to identify physicians
or to distinguish primary care providers from
specialists. Patient compliance is not tracked in the
system. Physician prescription orders are captured;
however, whether the prescription was filled
or refilled is not documented. Finally, the amount of
inpatient data is quite limited and would only be
there if documented by the physician in the outpatient
medical record. The University of Utah Pharmacotherapy
Outcomes Research Center (PORC) has been
involved in a number of studies using this EMR
database, taking advantage of the availability of lab
information. An initial study evaluated Body Mass
Index (BMI) as a side effect among different second
generation antipsychotics [1, 2]. A second
study determined the predictors of Chronic
Obstructive Pulmonary Disease (COPD) to assess
the impact of early diagnosis on outcomes [3]. A
final study conducted a risk assessment of OTC
non-steroidal anti-inflammatory drugs for perforations,
ulcers and gastro-intestinal bleeds [4]. Other
disease states such as metabolic syndrome and
its impact on chronic disease, blood pressure outcomes
related to mono and combination hypertension
therapy, and an assessment of mono vs.
combination therapy in the treatment of diabetes
via HbA1c outcomes are also well suited to the
EMR database and are currently under study. Canadian Data
The Canadian database prototype is provided by
British Columbia (BC), a province comprised of over
4.1 million residents located on the West Coast of
Canada. The dataset is population-based and
patient specific. It contains demographic information
about the entire population, as well as utilization
records for prescription drugs, medical, and hospital
services. The denominators are assembled by
the UBC Centre for Health Services and Policy
Research. The study cohort is every BC resident eligible
for the provincially administered, universal
public health insurance, excluding members of the
Royal Canadian Mounted Police, First Nations
(aboriginal peoples of BC), and Veterans, which
represent 4% of the entire population. Information on prescription drugs is provided by
the BC PharmaNet System, which is a computer
network linking all pharmacies in the province. The
primary use of this system is for patient-specific
drug interaction monitoring and insurance claim
adjudication. By law, it captures every prescription
dispensed in BC regardless of payment type (public
drug plan, private drug plan, or out-of-pocket).All individuals, regardless of age, are included in
this database. PharmaNet does not capture prescriptions
in acute care and rehabilitation hospitals,
or prescriptions for residents of penitentiaries. Diagnostic information of indicated and potentially
confounding conditions is captured through
Medical Services Plan and Hospital Discharge
records. The former pertains to outpatient fee-forservice
billings and contains one diagnosis per
physician visit or related service. The latter
describes patient stays at the point of hospital discharge,
and contains up to 16 diagnoses. The
team at the UBC Centre for Health Services and
Policy Research uses these diagnoses for
research purposes, identifying and aggregating
conditions using Expanded Diagnostic Clusters
(EDCs), a system developed by the Johns Hopkins
Adjudicated Clinical Groups (ACGs) team. The BC database has already been used to study a
wide range of health services research questions,
including numerous drug utilization reviews (hypertension
drugs, antidepressants, cholesterol agents,
asthma treatments) and drug policy analyses.
Using the BC databases, detailed information about
prescription drug utilization has been compiled in
the BC Rx Atlas, available online from the UBC
Centre for Health Services and Policy Research [5]. Italian Data
In collaboration with the Agenzia Sanitaria
Regionale of the Regione Emilia Romagna (RER)
of Italy (in Northeast Italy), the Center for Research
in Medical Education and Health Care at Jefferson
Medical College in Philadelphia, PA, USA, has constructed
a population-based, longitudinal health
care utilization database for the region. The nature
of the Italian health care system makes the resulting
database a valuable resource for planning and
research. Italy has a universal health care system
which is very similar to Great Britain. The RER database contains, therefore, records for the entire
population of the region, approximately 4 million
residents. The population of RER is increasing at a
very slow rate, owing to a positive but small net
migration rate that makes up for a low birth rate.
As a result, the population in RER database is
highly stable, yielding a dataset that is not subject
to problems of censored observations or other
statistical problems. This provides a great advantage
over other utilization databases.
The RER database includes demographic information
on all residents, including age, gender, birth
and death date, location of residence, primary care
physician, with linkages to information about the
resident’s primary care physician (a general practitioner
in Italy); day and ordinary hospital discharge
abstract data, including ICD-9_CM coded
diagnosis and procedure codes, admission and
discharge dates, and DRG-based payments for
RER residents hospitalized either in hospitals in
RER or in other regions of Italy; and outpatient
pharmacy data at the individual prescription level,
including drug codes, pharmacy payments, and
patient co-payments. There are also data describing
use and costs of specialty care (including lab,
diagnostics, therapeutic procedures, visits to specialists);
home health data (physician, nurse, therapist,
etc.); physician information (i.e. payments
received, specialty, years in practice, and patient
load); and ad hoc registries of specific medical
devices or surgical procedures. All of this information, fully linkable at both patient
and physician levels, currently is available for 5
consecutive years (2000-2004), and there is
ongoing data collection. The database can be
indefinitely updated with information from additional
years, enabling retrospective analyses
across an ever-increasing sample frame and
prospective analysis as new years of data are
added to the database.
The value of the database has been increased by
adding clinical classifications mapped from the
hospital and pharmacy data. Diagnostic codes
from day and acute hospital admissions have been
used to classify admissions using the Disease
Staging classification which, in turn, has been
used to identify the subset of individuals who may
be at higher risk for utilizing more extensive or
expensive health services in the future [6]. Another
set of indicators (Chronic Condition Drug Groups -
CCDGs) uses outpatient pharmacy data and the
Italian national formulary to identify individuals
with chronic disease [7]. Using both hospital and
pharmacy data, staging groups and CCDGs have
been used to identify chronic diseases in a body
system-etiology framework.
The database is being used in a variety of projects
including assessment of the appropriateness of
acute hospital admissions [8], evaluation of potentially
inappropriate prescribing patterns for patients
over age 65 [9], assessment of the impact of copayment
changes on pharmacy and hospital use,
and analyses of concentration and persistence of
cost in sub-populations of the region [10]. In addition,
the data are being used as a part of a “riskadjustment”
model that identifies individuals with
chronic disease and predicts future year costs
[11]. This information can then be used to rationalize
the allocation of resources to health districts
based on the needs of the local population.
 Similarities and Differences
among the Three Databases
There are important similarities and differences
among the three databases (Table 1). Of course
the most striking difference is that the GE
Centricity database is actually an EMR, which is a
rich source of data, used primarily by providers to
manage their outpatient encounters. The databases
in Canada and in Italy are claims data, used for
administrative purposes. The Canadian and the
Italian datasets are actually quite similar. In terms
of available sources, all of the different databases
have difficulties in getting specific inpatient data,
including pharmacy use. All have out-patient prescription
data, and the sources of individual
providers vary. The GE Centricity data is based on
practice groups and not individual practitioners.
Cost data is defined on the BC and Italian datasets.
The largest difference is the population described
by the data, although in all three datasets there are
roughly 4 million people. The GE dataset is scattered
over a large geographic area. Obviously, it is
more in-depth by being an electronic medical
record. The Canadian and Italian datasets include
data for a captured population. For morbidity case
identification and classification, all have ICD-9
codes and drug use, of course, but these codes
are used in different ways. For instance, in Canada
researchers are using the EDCs based on the Hopkins work, whereas at Jefferson, researchers
have developed the CCDGs with the Italian database.
From a research perspective, the detailed
clinical outcomes information in the GE Centricity
data is of great value. In the Canadian and Italian
datasets, claims data can be examined to detect
occurrences of adverse events in terms of hospitalization
or discontinued prescription use, which
cannot be assessed in the EMR. Challenges and Opportunities to
Conduct Multi-National
Retrospective Outcomes
Research Evaluations
Due to the various differences within the datasets
and across global markets, there are benefits as
well as challenges of using data from multiple
international sources. Using multiple datasets
from different countries enables the verification of
findings from one country based on information
from other countries. For example, suppose a relationship
between an adverse event and a drug
agent is identified in country A. There might be a
unique characteristic of the population that might
limit the credibility of the finding. However, if this
relationship also exists in other countries,
researchers and policy makers are more apt to
believe the finding. An additional benefit is the
opportunity to compare practice patterns across
global regions and countries. For instance, by
looking at differences on prescribing patterns of
certain medications across countries, we can
determine whether physicians in some countries
are better able to adhere to current guidelines than
physicians in other countries. This enables us to
identify potential best practices and policies. For
example, if the rate of adherence is greater in
some countries than others, and all those countries
have continuing medical education (CME)
focused exclusively on the guidelines, one might
hypothesize that the use of CME might be an
approach other countries should follow. However, making comparisons among databases
is not an easy task. First of all, one should always
ask whether or not each individual dataset has had
a formal evaluation of its data quality.
Administrative data, for example, sometimes has
problems in quality assurance. It is therefore
important to understand the data-generating
process, including the method of data collection
and the population from which the data is collected.
If the administrative data is used for reimbursement
purposes, like the United States Medicare
data, there are often government regulations to
check for upcoding, etc. Although the Italian database
is not used for reimbursement, the health
department in RER takes time to ensure data quality.
Other data systems may not have quality
assurance programs. For example, pharmaceutical
claims data may not have a reconciliation
process for re-submitted claims. As a result, one
would see multiple claims for the same prescription.
Bulk billing may make it difficult to link claims
with actual services. One major weakness is inevitable churning in and
out of commercial insurance based databases in
the United States. The United States has a rather
fragmented health insurance system so there may
be individuals moving in and out of the database.
The Italian population utilization database from RER
does not have this problem as the population is
extremely stable. The database captures utilization
of persons in the population until they die or leave
the region-which is somewhat rare, except for the
student population. Canada has a single payer,
national health insurance model, and the populations
in their databases are relatively stable as well.
The stability of the investigated population is important
when observing and comparing differences
across countries, though it can potentially be corrected
through the choice of statistical models. There is a further need to understand the structural
differences in health care provision across
countries in order to interpret cross-country differences.
Consider drug availability. Often, drugs are
introduced for the first time in the United States,
but it takes a while before they are introduced in
other countries, or vice versa. So it may not be
possible to compare outcomes on specific drugs
until the drug has been around for awhile. This
may also explain why certain physicians in one
country adopt a new procedure or certain new
drug agents sooner than another. This can be
ascertained through making these cross-country
comparisons. It is very important not only to understand the context,
but also to recognize that there will be different
error structures in the data as a result of the
differences in the data generating processing. And
so, it is important to utilize several different measures,
or “indicators,” instead of using just one
measure to look at an outcome. It is also useful to
use ratios and to compare trends rather than just
using simple numbers. For example, instead of
looking at differences in volume across countries,
sometimes it is useful to look at the volume per
person across different countries. If there is interest
in the volume of a new antihypertensive medication
you can compare volume trends in the new
antihypertensive across two countries. This may
require interpretation in two parallel lines to indicate
no real difference in uptake between the two
countries. However, a better analysis would compare
the ratio of the volume of the new antihypertensive
to the volume of the standard hypertensive
medication. Even if volume trends are parallel, the trends in the ratio may not be. For example, the
ratio may increase faster in one country compared
to another. This would indicate that, even though
volume was increasing at the same rate for countries
A and B, there was a greater level of substitution
in country B from the old to the new hypertensive
medication. Thus, wherever possible, it is
important to look at trends using ratios. In looking at cost data, it is also important to make
sure that a common currency is used. Deflate the
data to use a common year, and then use purchasing
power parity so that the same buying power
throughout the different countries can be seen. And finally, there should be consideration of various
specification tests particularly in multivariate
models. For example, one can estimate a model for
each country in a regression and perform something
like a Chow test to determine whether or not
there are structural differences across countries. Conclusions
Conducting multi-national retrospective outcomes
research is appealing. Certainly, there are numerous
challenges to be overcome, yet the potential
information that can be drawn from such studies
is of great value. Therefore, we strongly encourage
institutions and organizations to increase the sharing
of data in order to understand better the population
needs in different countries. This will further
assist in measuring important cross national outcomes
and setting global health care policy
accordingly.
REFERENCES
1. Brixner DI, Said Q, Newcomer J, et al. Real world usage of weight gain
drugs in patients on second-generation anti-psychotics. American
Diabetes Association. San Diego, CA, June 10-14th, 2005.
2. Brixner DI, Blonde L, Said Q, et al. Real World Impact of Second
Generation Antipsychotics on Weight Gain. American Diabetes
Association, San Diego, CA, June 10-14th, 2005.
3. Joish VN, Brady E, Stockdale WA, et al. Identification and diagnosis
of COPD in a primary care setting. International Conference American
Thoracic Society; San Diego, CA, 2005.
4. Biskupiak JE, Brixner DI, Oderda GM, Howard K. Gastrointestinal
Complications of over-the-counter non-steroidal anti-inflammatory drugs.
ACG 70th Annual Scientific Meeting; October Honolulu, HI, 2005.
5. UBC Centre for Health Services and Policy Research web site.
Publications. Available at:
http://www.chspr.ubc.ca Last Accessed
February 16, 2006.
6. Louis DZ, Gonnella JS, McCracken SB, Barnes CA. Patient classification
and severity of illness: Disease Staging, DRGs, and APR-DRGs.
Patient Classification Systems/Europe, Budapest, Hungary, October 2004.
7. Maio V, Yuen EJ, Rabinowitz C, et al. Using Pharmacy Data to Identify
Those with Chronic Conditions in Emilia Romagna, Italy. J Health Serv Res
Policy 2005;10:232-8.
8. Louis DZ, Donatini A, Jimbo M, et al. An atlas of appropriateness of
hospital admissions in the Emilia-Romagna Region, Italy. 132nd American
Public Health Association, Washington, DC, November 2004.
9. Maio V, Yuen EJ, Novielli KD, et al. Potentially Inappropriate Prescribing
for Elderly Ambulatory Patients in Regione Emilia Romagna, Italy. 10th
Annual Meeting of the International Society Pharmacoeconomics and
Outcomes Research, Washington, DC, May 16, 2005,
10. Yuen EJ, Maio V, Louis DZ, et al.Using an Italian Population Database
to Profile Prevalence and Costs of Chronic Conditions. 8th Annual
European Congress of the International Society Pharmacoeconomics and
Outcomes Research, Florence, Italy, November 8, 2005.
11. Smith KD, Yuen EJ, Donatini A, Louis DZ, Rabinowitz C, Maio V, Taroni
F. Risk Adjustment in a Non-Market-Based System: The Case of Emilia
Romagna, Italy. Intl J Healthcare Technol Managt 2006;7:100-16.
|
