Where Does Outcomes Research Fit into Comparative Effectiveness Research? How is Comparative Effectiveness Research Used in Evidence-Based Decision-Making

THE ISSUE:

Evidence based medicine (EBM) activities in operation today do not necessarily take into account outcomes research relevant to the questions they are trying to answer. In some cases, e.g. the Oregon Drug Effectiveness Review Project (DERP), the entire EBM process consists of a systematic review of RCT data of drugs within class. In other cases, e.g. the AMCP Format, a much broader concept of evidence is used, including virtually all aspects of HEOR. To the extent that outcomes research is not included in evidence-based decision making, it calls into question the usefulness and viability of the field of outcomes research.

EBM is becoming a national issue within Medicaid, CMS and consumer organizations, including Consumers Union and AARP. The OHSU DERP initiative is sweeping the country and defining EBM as solely a review of RCTs to the exclusion of outcomes research. This should be considered a threat to ISPOR’s mission

ISSUE DESCRIPTION:

• The idea that health care decisions should be based on best evidence is unassailably wise and not controversial. One prominent example of formalizing the process for synthesizing clinical evidence for use in health care decision-making¹ is the Cochrane Collaboration, an international non-profit and independent organization that produces and disseminates systematic reviews of healthcare interventions and promotes the search for evidence in the form of clinical trials and other studies of interventions².
  
  
• Ideally, such systematic review results in a much better assessment of the collective findings of the most rigorous clinical literature, providing a higher level of confidence that this literature is interpreted correctly. Although the systematic review process has been well-accepted (at least within academic settings) as methodologically sound and useful for guiding clinical policy, there is less evidence that it has been incorporated into physician-patient decision-making,³ its initial raison d’etre.
  
  
• Recently, there has been a movement in the U.S. and elsewhere to incorporate a systematic review process into health care decision-making in a broader fashion, for instance for national or state Medicaid authorities or managed care P&T committees to make decisions as to which medical products and/or services are to be covered or reimbursed for whole populations. This movement has given rise to the terms “evidence-based medicine” (EBM) and “evidence-based decision-making” as practiced, for example, by the state of Oregon Health Plan and the Oregon Health & Science University’s Evidence-Based Practice Center (EPC) in developing a preferred drug list (PDL).
  
  
• When EBM is the basis for population decision-making, it necessarily loses focus on health consequences related to individual patients, to patient-specific idiosyncrasies and patient preferences. These weaknesses are particularly germane and are (or should be) of concern to the outcomes research community, patients and their families, practitioners and many others who value fully-informed decision-making because they address the core of what outcomes research is intended to bring to patient welfare.

ISSUE ANALYSIS:

• Defining evidence requires focus on measurement of health outcomes. A narrow interpretation of EBM results in use of only “statistically significant” (e.g., p<0.05) findings derived from randomized clinical trials. This ignores an enormous amount of scientific evidence from broader but still scientifically rigorous outcomes research.
  
  
• For instance, in addition to best efficacy evidence, outcomes researchers are interested in such health and economic outcomes as: 1) average effectiveness (as opposed to efficacy) of treatments due to, for instance, community practice patterns and patient adherence; 2) individual preferences, symptom control and impact on health-related quality of life, for instance due to the effectiveness or side-effect characteristics of a drug, or a more convenient regime; 3) estimated long-term outcomes for, say, chronic medications, which commonly extend months or years beyond clinical trial data; 4) cost-effectiveness information to help decision-makers determine best value for money; and 5) budget impact analysis to help decision-makers evaluate whether they can afford the addition of a new technology.
  
  
• Many groups in the US and Europe already include broader health outcomes in their evaluation of health care evidence, including, the Centers for Disease Control and Prevention (CDC), the Agency for Health Research and Quality (AHRQ), the Centers for Medicare and Medicaid, researcher organizations such as the Society for Medical Decision-Making (SMDM), the International Society of Pharmacoeconomics and Outcomes Research (ISPOR) and Academy Health. The Academy of Managed Care Pharmacy (AMCP) has expressly come out with the Format for Formulary Submissions⁴, which calls on its managed care pharmacy constituents to include a broad array of clinical, economic and outcomes evidence into account for formulary decision-making. These efforts and positions are mirrored around the westernized world by organizations such as the U.K.’s National Institute for Clinical Excellence (NICE), the Canadian Coordinating Office of Technology Assessment (CCOHTA), Australian’s Pharmacy Benefit Advisory Committee (PBAC), and numerous international medical technology assessment organizations including International Network of Agencies for Health Technology Assessment (INAHTA)
  
  
• Is practiced by EBM purists, the only evidence that will be considered is that which is derived from randomized clinical trials (RCT) and meets the traditional “statistically-significant” (i.e. 0.05 cutoff) level as derived from a classical statistical analytical approach (the exception, at least in the Oregon case, seems to be safety information, which is accepted from non-randomized studies). Otherwise suggestive evidence derived from less traditional approaches, including Bayesian analyses of data, decision modeling and analyses from database studies are not considered at all.

REFERENCES

1. “Effectiveness and efficiency: random reflections on health services” Archie Cochrane
2. Cochrane Collaboration, www.cochrane.org
3. ”What kind of evidence is it that Evidence-Based Medicine advocates want health care providers and consumers to pay attention to?” R.B. Haynes, BMC Health Servc. Res. 2 (1) (2002) 3.
4. Format for Formulary Submissions, Version 2.0., October 2002

Task Forces Index | Health Science Council