Introduction
A number of recent and well-publicized events involving withdrawals of drugs from the U.S. market[1, 2] have underscored the importance of under-standing and measuring individuals’ preferences regarding risk-benefit trade-offs. In all of these cases, interventions offering potentially significant therapeutic benefits were found to carry increased risks of serious and often life-threatening adverse events. Decisions to halt the development or sale of these therapies clearly require balancing of benefits and risks; however, establishing criteria for when the risks outweigh the benefits present important challenges to decision makers.

These highly publicized pharmaceutical product withdrawals call attention to the fact that our scientific information base at the time of product approval is imperfect. Pharmacoepidemiologists and drug-safety professionals are well aware of the shortcomings of that information base: insufficient numbers of homogeneous patients monitored for too short a time, under conditions of an experiment rather than clinical practice. Regulatory agencies have recognized that all products need continued safety vigilance. FDA and EMEA require special management for some products to assure that the treatment benefits outweigh risks in the clinical setting [3 - 9].However, related guidance documents focus primarily on quantitative clinical evidence of risks rather than the risk-benefit balance, and generally ignore values of patients, physicians or other stakeholders.

A review of past examples of product withdrawals and risk-management decisions across multiple countries reveals that decisions often are inconsistent, and are based on very limited scientific information beyond the original clinical trial data on safety and efficacy [10]. Sometimes those decisions are informed by advisory panels of scientists and clinicians. Occasionally those decisions are influenced by patient and physician stakeholders. For example, HIV patients demanded early access to experimental antiviral treatments two decades ago. Public testimony at FDA advisory committee meetings have presented forceful personal demands either for expanded access to medications, as in the case of with-drawn products for irritable bowel syndrome and multiple sclerosis [11, 12], or for greater warnings or restricted use, as in the case of antidepressants thought to increase the potential for suicidal ideation and behavior in some pediatric patients [13]. No systematic method for eliciting and evaluating stakeholder values on risks and benefits has emerged from the recent round of regulatory guidance or from other public debate.

While it is important to hear the testimony of individuals who genuinely believe that the benefits of these products outweigh the risk of infrequent, but sometimes deadly, adverse events, such testimony does not provide systematic, quantifiable evidence of the willingness of stakeholders to accept observed levels of risk in order to achieve the therapeutic benefits of these products. It is appropriate to question whether those who expressed demands for greater access to medications reflect the population for whom the medication is indicated. What is needed is a valid method for assessing the preferences of a more representative sample of the patient population and a means of incorporating such preferences in risk-benefit decision rules. To quantify patients’ preferences for the benefits and risks associated with any pharmacotherapy, it is important to con-sider the following questions:

1. What are the salient treatment features that are relevant for evaluating pharmaceutical benefits and risks in regulatory decision making?

2. What is the relevant population to whom the benefits and risks of treatment accrue?

3. Are there valid methods for measuring the relative importance of benefits and risks to decision makers in a common metric?

4. The first two questions relate to defining the risk-benefit problem; that is, who is affected by the regulatory decision and how. The third question is concerned with identifying a method that will provide useful information to decision makers.

Two Methods for Risk-Benefit Analysis
Two methods have been used to systematically demonstrate the willingness of stakeholders to trade off risks for benefits. Both of these methods were discussed in an issues panel during the ISPOR 11th Annual International Meeting [14 -16]. These approaches include incremental net health benefits (INHB) and maximum acceptable risk (MAR) analyses.

Incremental Net Health Benefits (INHB)
Lynd [16] has recently proposed a framework for evaluating the risks and benefits of pharmaceutical treatments known as the incremental net health benefits (INHB) approach. Benefits and adverse events associated with a treatment are quantified using available clinical trial or post-marketing surveillance data. Importance weights, usually health-state utilities, are assigned to each outcome in order to express all benefits and all risks in a common metric. The difference between the sum of the weighted benefits and the sum of the weighted risks of a treatment thus rep-resent the net health benefits (NHB) of the treatment. INHB is then calculated as the difference between the NHB of the treatment of interest less the NHB of an alternative treatment or standard of care. A positive INHB indicates that the net benefits of treatment are positive relative to an appropriate comparator.

Studies using this type of method to evaluate treatment risks and benefit include Sutton et al. [17] and Minelli et al. [18]. Sutton and colleagues esti-mated the incremental net health benefits using warfarin to reduce the risk of embolic stroke relative to the risk of fatal hemorrhage in patients with non-rheumatic a trial fibrillation. Minelli and colleagues developed a model to evaluate the benefits and risks associated with hormone replacement therapy in women with varying risks of breast cancer. The implication of the results of each of these studies is that if the net health benefits of the treatment are positive, then the treatment should be approved for use, even if the treatment poses rare but significant safety concerns.

Maximum Acceptable Risk (MAR)
A somewhat different approach has been proposed by Johnson [15]. The objective of this approach is to estimate the maximum risk patients are will-ing to accept in order to achieve the therapeutic benefits of a pharmacotherapy that incorporates patient preferences over both risk and benefit out-comes as well as over different risk levels. This maximum acceptable risk (MAR) then can be compared against the actual or expected risk associated with a treatment to determine whether the treatment is acceptable to patients.

MAR is estimated using choice-experiment or conjoint-analysis methods - an increasingly common method of patient preference analysis in health economics [19] - in which decision makers are asked to choose between two or more alternative hypothetical treatment options. Each treatment option is defined by levels of different treatment attributes that can include efficacy and safety outcomes associated with the treatment. When evaluating the relative importance of risks and benefits, the levels of each treatment attribute represent either the severity of the treatment outcome or the likelihood that the efficacy or adverse-event outcome will occur. MAR is calculated from preference parameters estimated from the resulting choice data as the increase in risk that exactly offsets the increase in patient satisfaction from greater treatment efficacy. In contrast to the INHB approach, this approach thus elicits patients’ subjective assessment of the risks and benefits directly. MAR is the patients’ own appraisal of the risk level at which the net benefits are zero.

An Empirical Example of MAR
Crohn’s disease (CD) is a chronic, relapsing inflammatory disease of the gastrointestinal tract, characterized by symptoms of abdominal pain, diarrhea, and rectal bleeding. Serious complications of CD include fistulas, abscesses, bowel obstruction, and anal fissures that may require repeated surgeries and resections. The symptoms, complications, and comorbidities of CD often result in poor health-related quality of life and limit patients’ ability to participate in normal physical and social activities.

Clinical trials have demonstrated significant benefits of natalizumab( TYSABRI®) for many patients with CD [20]. The risk of developing serious side effects, however, is an important concern for many CD treatments. The main risk of concern associated with natalizumab is the possibility that patients taking the drug will have a greater chance of developing progressive multifocal leukoencephalopathy (PML), a rare but usually fatal central nerv-ous system infection. Three cases of PML have been diagnosed in patients exposed to natalizumab. Approximately 1,200 patients were treated withn a talizumab in CD clinical trials and another 1,600 patients received natalizumab in clinical trials for multiple sclerosis (MS). For other CD treatments, the main serious side effect risks of concern include lymphoma and serious infections such as tuberculosis and pneumonia.

In a recent choice-format CA survey [21], we surveyed 570 CD patients using a web-enabled survey instrument. The survey described seven primary treatment attributes: four measures of treatment efficacy and three measures of serious side effect risks. The efficacy attributes included measures of: 1)CD symptoms and severity levels; 2) the frequency of flare-ups; 3) prevention of serious complications from CD; and 4) the need for oral steroids.

The three serious side effect attributes were: 1) the risk of death or severe disability from PML; 2) the risk of death from serious infections; and 3) the risk of death from lymphoma. The severe disability outcome was included in the PML definition to recognize that potential and because it has been noted that for many persons, severe disability may be perceived as a worse out-come than death. For each of the three side-effect attributes, four potential levels of risk were described, ranging from 0% to 5% probability of experiencing the serious side effect within the next 10 years.

The following figure shows the maximum acceptable 1-year risk of the three potential serious adverse events for different improvements in symptom severity, holding all other treatment attributes constant. In each case, the MARs for all serious adverse events are likely to exceed the observed rate of these adverse events. Therefore, it is reasonable to conclude that many patients who are fully informed as to the potential risks of a treatment are willing to accept these risks in order to receive the benefits of treatment.

Conclusions
Recent withdrawals and reintroduction of drugs in the U.S. market have underscored the need for a systematic framework for demonstrating the relative importance of risks and benefits of pharmaceuticals. Recent guidance from regulators emphasizes the use of quantitative clinical evidence of risk, but does not address the value of therapeutic benefits to patients, physicians or other stakeholders.

MAR and INHB facilitate comparing the risks and benefits of new and existing treatments. State-of-the-art methods can provide rigorous estimates of the rates at which patients are willing to trade risks for improvements in clinical outcomes. These studies can be conducted at any stage in a product’s development, and can be used to evaluate the importance of signals that develop at any stage in the product lifecycle.

The INHB approach applies importance weights to the likelihood of each benefit and risk outcome to develop a single measure of the net health benefits of a treatment. The MAR approach provides an estimate of the zero net benefit thresholds by estimating the maximum risk patients would be willing to accept in order to achieve a specified therapeutic benefit. Although these two methods approach the risk-benefit tradeoff problem somewhat different-ly, they share a common concern with using a common metric to directly compare negative and positive consequences of treatments. It is possible that these methods can be combined into a comprehensive framework in which conjoint utility estimates that evaluate the explicit tradeoffs between benefits and risks of a therapy and reflect risk aversion, can be used as weights in the INHB framework.

Because these approaches to risk-benefit analysis are relatively new, there are inevitably questions that will arise regarding their appropriate use. However, consensus as to the appropriate role of quantified patient preferences in risk-benefit analysis is likely to emerge as we gain experience in applying these methods in various regulatory settings

References

1. Wysowski DK and Swartz L. Adverse drug event surveillance and drug withdrawals in the United States,1969-2002. Arch Intern Med 2005;165:1363-69.

2. Singer E. Tysabri withdrawal calls entire class into question. Nature Medicine 2005;4:359.

3. EMEA Guidance on Risk Management Systems for Medicinal Products for Human Use. November 14,2005. Last accessed 28 Feb 2006 at: http://www.emea.eu.int/pdfs/human/euleg/9626805en.pdf

4. FDA Guidance for industry: development and use of risk minimization action plans. March 2005. Last accessed April 6, 2005 at: http://www.fda.gov/cder/guidance/6358fnl.htm

5. FDA Guidance for industry: Good Pharmacovigilance Practices and Pharmacoepidemiologic Assessment.March 2005. Last accessed April 6, 2005 at: http://www.fda.gov/cder/guidance/6359OCC.htm

6. FDA Guidance for industry: Premarketing Risk Assessment March 2005. Last accessed April 6, 2005 at:http://www.fda.gov/cder/guidance/6357fnl.htm.

7. Food and Drug Administration Center for Drug Evaluation and Research. Summary Minutes of thePsychopharmacologic Drugs Advisory Committee Meeting and the Pediatric Subcommittee of the anti-infective drugs advisory committee, February 2, 2004. Last accessed May 9, 2006 at:http://www.fda.gov/ohrms/dockets/ac/04/minutes/4006M1_Final.htm.

8. Food and Drug Administration Center for Drug Evaluation and Research Peripheral and Central Nervous System Drugs Advisory Committee, Volume I, Tuesday, March 7, 2006. Last accessed May 10, 2006 at:http://www.fda.gov/ohrms/dockets/ac/06/transcripts/2006-4208T1.pdf.

9. Food and Drug Administration Gastrointestinal Drugs Advisory Committee with the Risk Management Subcommittee of the Advisory Committee for Pharmaceutical Science, April 23, 2002. Open Public Hearing Materials. Last accessed May 9, 2006 at:http://www.fda.gov/ohrms/dockets/ac/02/briefing/3848b1_oph.htm.

10. Hirst C, Cook S, Dai W, Perez-Gutthann S, Andrews E. A Call for International Harmonization in Therapeutic Risk Management. (in press) Pharmacoepidemiology and Drug Safety.

11. Mathews AW, Westphal SP. Tricky FDA Debate: Should a Risky Drug Be Approved Again?, The Wall Street Journal. February 24, 2006.

12. Armstrong D. Research Bolsters Tarnished MS Drug. The Wall Street Journal. August 10, 2005.

13. Grady D, Gardiner H. Over prescribing Prompted Warning on Antidepressants, The New York Times, March 24, 2004.

14. Garrison L. Assessing the Potential Value of Pre-Marketing Quantitative Risk-Benefit Modeling of New Pharmaceuticals. Issues Panel: Health outcomes approaches to risk-benefit analysis: how ready are they? 11th International Meetings of the International Society for Pharmacoeconomics and Outcomes Research. May 2006. Philadelphia, PA.

15. Johnson FR. Returning to the Fold: Conventional Utility versus Health Utility for Risk-Benefit Analysis. Issues Panel: Health outcomes approaches to risk-benefit analysis: how ready are they? 11th International Meetings of the International Society for Pharmacoeconomics and Outcomes Research. May 2006.Philadelphia, PA.

16. Lynd L. Quantitative Methods for Therapeutic Risk-Benefit Analysis. Issues Panel: Health outcomes approaches to risk-benefit analysis: how ready are they? 11th International Meetings of the International Society for Pharmacoeconomics and Outcomes Research. May 2006. Philadelphia, PA.

17. Sutton AJ, Cooper NJ, Abrams KR, et al. A Bayesian approach to evaluating net clinical benefit allowed for parameter uncertainty. J Clin Epidemiol 2005;58:26-40.

18. Minelli C, Abrams KR, Sutton AJ, Cooper NJ. Benefits and adverse events associated with hormone replacement therapy: clinical decision analysis. BMJ 2004;328:371-6.

19. Ryan M, Scott DA, Reeves C, et al. Eliciting public preferences for healthcare: a systematic review of techniques. Health Technol Assess 2001;5:1-186.

20. Sandborn WJ, Colombel JF, Enns R, et al. Natalizumab induction and maintenance therapy for Crohn’s disease: The Enact-1 and Enact-2 Trials, New Eng J Med 2005;353:1912-25.

21. Johnson FR, Ozdemir S, Mansfield C, et al. Crohn’s disease patients’ benefit-risk preferences: serious adverse event risk versus treatment efficacy. 2006. Data on File. Elan Pharmaceuticals, San Diego, CA.