Evidence-based Medicine & Outcomes Research Task Force

WHERE DOES OUTCOMES RESEARCH FIT INTO EVIDENCE-BASED HEALTH CARE DECISION-MAKING?

THE ISSUE:

Evidence based medicine (EBM) activities in operation today do not necessarily take into account outcomes research relevant to the questions they are trying to answer. In some cases, e.g. the Oregon Drug Effectiveness Review Project (DERP), the entire EBM process consists of a systematic review of RCT data of drugs within class. In other cases, e.g. the AMCP Format, a much broader concept of evidence is used, including virtually all aspects of HEOR. To the extent that outcomes research is not included in evidence-based decision making, it calls into question the usefulness and viability of the field of outcomes research.

EBM is becoming a national issue within Medicaid, CMS and consumer organizations, including Consumers Union and AARP. The OHSU DERP initiative is sweeping the country and defining EBM as solely a review of RCTs to the exclusion of outcomes research. This should be considered a threat to ISPOR’s mission

ISSUE DESCRIPTION:

The idea that health care decisions should be based on best evidence is unassailably wise and not controversial. One prominent example of formalizing the process for synthesizing clinical evidence for use in health care decision-making1 is the Cochrane Collaboration, an international non-profit and independent organization that produces and disseminates systematic reviews of healthcare interventions and promotes the search for evidence in the form of clinical trials and other studies of interventions.²
Ideally, such systematic review results in a much better assessment of the collective findings of the most rigorous clinical literature, providing a higher level of confidence that this literature is interpreted correctly. Although the systematic review process has been well-accepted (at least within academic settings) as methodologically sound and useful for guiding clinical policy, there is less evidence that it has been incorporated into physician-patient decision-making,³its initial raison d’etre.
Recently, there has been a movement in the U.S. and elsewhere to incorporate a systematic review process into health care decision-making in a broader fashion, for instance for national or state Medicaid authorities or managed care P&T committees to make decisions as to which medical products and/or services are to be covered or reimbursed for whole populations. This movement has given rise to the terms “evidence-based medicine” (EBM) and “evidence-based decision-making” as practiced, for example, by the state of Oregon Health Plan and the Oregon Health & Science University’s Evidence-Based Practice Center (EPC) in developing a preferred drug list (PDL).
When EBM is the basis for population decision-making, it necessarily loses focus on health consequences related to individual patients, to patient-specific idiosyncrasies and patient preferences. These weaknesses are particularly germane and are (or should be) of concern to the outcomes research community, patients and their families, practitioners and many others who value fully-informed decision-making because they address the core of what outcomes research is intended to bring to patient welfare.

ISSUE ANALYSIS:

Defining evidence requires focus on measurement of health outcomes. A narrow interpretation of EBM results in use of only “statistically significant” (e.g., p<0.05) findings derived from randomized clinical trials. This ignores an enormous amount of scientific evidence from broader but still scientifically rigorous outcomes research.
For instance, in addition to best efficacy evidence, outcomes researchers are interested in such health and economic outcomes as: 1) average effectiveness (as opposed to efficacy) of treatments due to, for instance, community practice patterns and patient adherence; 2) individual preferences, symptom control and impact on health-related quality of life, for instance due to the effectiveness or side-effect characteristics of a drug, or a more convenient regime; 3) estimated long-term outcomes for, say, chronic medications, which commonly extend months or years beyond clinical trial data; 4) cost-effectiveness information to help decision-makers determine best value for money; and 5) budget impact analysis to help decision-makers evaluate whether they can afford the addition of a new technology.
Many groups in the US and Europe already include broader health outcomes in their evaluation of health care evidence, including, the Centers for Disease Control and Prevention (CDC), the Agency for Health Research and Quality (AHRQ), the Centers for Medicare and Medicaid, researcher organizations such as the Society for Medical Decision-Making (SMDM), the International Society of Pharmacoeconomics and Outcomes Research (ISPOR) and Academy Health. The Academy of Managed Care Pharmacy (AMCP) has expressly come out with the Format for Formulary Submissions⁴, which calls on its managed care pharmacy constituents to include a broad array of clinical, economic and outcomes evidence into account for formulary decision-making. These efforts and positions are mirrored around the westernized world by organizations such as the U.K.’s National Institute for Clinical Excellence (NICE), the Canadian Coordinating Office of Technology Assessment (CCOHTA), Australian’s Pharmacy Benefit Advisory Committee (PBAC), and numerous international medical technology assessment organizations including International Network of Agencies for Health Technology Assessment (INAHTA)
As practiced by EBM purists, the only evidence that will be considered is that which is derived from randomized clinical trials (RCT) and meets the traditional “statistically-significant” (i.e. 0.05 cutoff) level as derived from a classical statistical analytical approach (the exception, at least in the Oregon case, seems to be safety information, which is accepted from non-randomized studies). Otherwise suggestive evidence derived from less traditional approaches, including Bayesian analyses of data, decision modeling and analyses from database studies are not considered at all.

RECOMMENDED ISPOR ACTIONS:

Recruit a senior advisory panel to consist of both traditional clinical EBM experts (e.g. Cochrane participants) and HEOR researchers to recommend the objective to be achieved, to help define the problem and approach and to oversee the work plan and work products
Review, describe and report on existing EBM applications by key organizations in US and the rest of the westernized world
Review and report on the EBM methods literature including how various parties define the of the words “evidence-based medicine”, “best evidence”, “systematic review” with the intention of defining these words for different contexts.
Develop a consensus for the definitions of words and phrases above
Review and report on methods for combining disparate sets of evidence that include RCT evidence and outcomes evidence
Convene a 2 day workshop of key opinion leaders who have different concepts of the EBM application and charge the group with engaging the issues and developing a consensus concerning the role that HEOR (including observational data, modeling, patient-reported outcomes, including patient preference) should play in different applications of EBM and the methodological and reporting solutions.
Finalize white paper for circulation, comment and publication

REFERENCES

“Effectiveness and efficiency: random reflections on health services” Archie Cochrane
Cochrane Collaboration, www.cochrane.org
”What kind of evidence is it that Evidence-Based Medicine advocates want health care providers and consumers to pay attention to?” R.B. Haynes, BMC Health Servc. Res. 2 (1) (2002) 3.
Format for Formulary Submissions, Version 2.0., October 2002

Task Forces Index | Health Science Council