Budget Impact Analyses Get Some Respect
Abstract
Budget impact analyses (BIA) have taken a back seat to cost-effectiveness analysis (CEA) in economic evaluation over the years. If CEA was the well-born and highly regarded member of the family, BIA was the slightly disreputable one, hanging around with a somewhat suspicious crowd of health-care managers, bureaucrats, budget officials, and other silo dwellers. There is no listing at all for “budget impact analyses” in the index of the US Panel on Cost-Effectiveness [1]. Other reference standards, including the well-known work by Drummond et al., give BIAs scant attention [2].
Reasons for its “dissing” are not hard to understand. Cost-effectiveness analyses are rooted in economic theory. They can help guide resource allocation decisions in a rational and consistent fashion. CEAs can aid policymakers as they attempt to optimize population health subject to resource constraints. Other forms of economic evaluation, such as cost-benefit analyses, in which benefits and costs are monetized, also yield a decision rule and have a strong basis in welfare economics. Even cost-minimization and cost-consequences analyses speak to the cost and benefit sides of the equation.
In contrast, BIA say nothing explicitly about health benefits. There is no optimization, no decision rule, no calculation of value, and no grounding in economic theory. They simply quantify the financial consequences of using health-care services (though they may consider benefits indirectly to the extent they capture cost offsets).
Yet, BIA have stubbornly refused to fade away. Indeed, payers worldwide keep demanding them. The reasons for the popularity are also not hard to fathom. Unlike CEAs, BIA speak to affordability, the chief concern of health managers everywhere. Like any shopper on a spree, the payer has a real budget constraint and worries that he could go broke buying all of the items purporting to deliver good value for money. Abstractions like cost per QALY calculations, though they provide a useful guide to value, have never addressed the manager’s overarching fiscal problem.
To date, however, BIAs have had few journal articles devoted to their use. Moreover, they have not received the formal guideline treatment that CEA has enjoyed. Thus an ISPOR Task Force Report on the topic is a welcome addition [3].
As the Report argues, a comprehensive assessment of a new health-care intervention at time of launch can benefit from both CEA and BIA. At least in some respects, the two approaches can be viewed as complementary. CEAs evaluate costs and outcomes associated with health-care strategies and can be used to guide coverage or reimbursement decisions. BIAs are used, as the Report notes, for budget planning, for forecasting, and for computing the impact of a health technology on health insurance premiums. BIAs can be helpful in estimating the financial consequences of adoption and diffusion and in predicting how a change in mix of drugs versus other therapies will impact the trajectory of health spending on a condition.
The publication of this Task Force Report will help legitimize the practice. The Task Force has performed its job well and delivered a balanced and thoughtful statement. The call for standards and rigor should help strengthen the field.
Importantly, there is a central focus on the practical needs of the decision-maker, which sets it apart from most reports on CEAs. The recommended perspective is that of the budget holder. The Report notes that the drawing of budget boundaries is a highly local exercise, and that the specific set of assumptions of interest should reflect the decision-maker rather than a “reference case.” The Task Force highlights the fact that analyses should include data that reflect the size, characteristics, and symptoms of the population at hand, as well as the current and new treatment mix. The Report emphasizes that where possible BIAs should incorporate decision-makers’ own populations for other parameter estimates.
There are some important qualifiers here as well, for example, that the purpose of BIAs is not to produce exact estimates but to provide a valid computing framework on possible budget consequences. The Report also usefully recommends “that the analyst use the simplest design that will generate credible and transparent estimates,” a kind of Occam’s razor for the field. There is also useful guidance for the reporting of analyses.
One only wishes the Report had gone further in a few respects. Additional discussion about how BIA are being used—and should be used—would have been helpful. One would have appreciated a worked example, especially for BIAs that attempt to measure cost-offsets associated with a new drug, rather than a mere quantification of the financial consequences of uptake. In fairness, this would have been a considerable undertaking and probably beyond the scope of any task force purview, but this kind of illustrative example was used to great effect by the Panel on Cost-Effectiveness and is something for ISPOR to consider for a future project.
One also would have liked more support for the basic assertion that BIAs is an “essential” part of a comprehensive economic assessment of medical technology. The Report notes that a technology could be efficient but not affordable, and that “in such instances, there is, unfortunately, no current scientific guidance on how to resolve this dilemma.” More commentary on this fundamental question, as well as some illustrative examples and discussion of a future research agenda would have been helpful. Also, the Report emphasizes that models must meet the needs of decision-makers but this begs many questions about what decision-makers really want and how anyone is to know.
A final note: it did occur to this reviewer that budget impact analyses have the unfortunate acronym “BIAS.” One assumes that that is an unhappy coincidence but it does provoke a point: there is concern as always about hidden and optimistic assumptions. The larger wonder, however, is why the drug industry is asked to do the work that health plans should be doing themselves.
These remarks should not detract from the good work of the group. ISPOR Task Force reports have earned a place as well-cited products that help codify and clarify important issues for the field, from the role of modeling to the use of economic evaluation alongside randomized clinical trials [4,5]. They serve as a focal point for reference and further debate. The Task Force Report on Budget Impact Analysis clearly follows in this tradition.
References
1 Gold MR, Siegel JE, Russell LB, Weinstein MC. Cost-Effectiveness in Health and Medicine. Oxford: Oxford University Press, 1996.
2 Drummond MF, Sculpher MJ, Torrance GW, et al. Methods for the Economic Evaluation of Health Care Programmes (3rd ed.). Oxford: Oxford University Press, 2005.
3 Mauskopf JA, Sullivan SD, Annemans L, et al. Principles of good practice for budget impact analysis: report of the ISPOR task force on good research practice—budget impact analysis. Value Health 2007;DOI: 10.1111/j.1524-4733.2007.00187.x.
4 Weinstein MC, O’Brien B, Hornberger H, et al. Principles of good practice for decision analytic modeling in health-care evaluation: report of the ISPOR task force on good research practices—modeling studies. Value Health 2003;6:9–17.
5 Ramsey S, Willke R, Briggs A, et al. Good research practices for cost-effectiveness analysis alongside clinical trials: the ISPOR RCT-CEA task force report. Value Health 2005;8:521–33.
Reasons for its “dissing” are not hard to understand. Cost-effectiveness analyses are rooted in economic theory. They can help guide resource allocation decisions in a rational and consistent fashion. CEAs can aid policymakers as they attempt to optimize population health subject to resource constraints. Other forms of economic evaluation, such as cost-benefit analyses, in which benefits and costs are monetized, also yield a decision rule and have a strong basis in welfare economics. Even cost-minimization and cost-consequences analyses speak to the cost and benefit sides of the equation.
In contrast, BIA say nothing explicitly about health benefits. There is no optimization, no decision rule, no calculation of value, and no grounding in economic theory. They simply quantify the financial consequences of using health-care services (though they may consider benefits indirectly to the extent they capture cost offsets).
Yet, BIA have stubbornly refused to fade away. Indeed, payers worldwide keep demanding them. The reasons for the popularity are also not hard to fathom. Unlike CEAs, BIA speak to affordability, the chief concern of health managers everywhere. Like any shopper on a spree, the payer has a real budget constraint and worries that he could go broke buying all of the items purporting to deliver good value for money. Abstractions like cost per QALY calculations, though they provide a useful guide to value, have never addressed the manager’s overarching fiscal problem.
To date, however, BIAs have had few journal articles devoted to their use. Moreover, they have not received the formal guideline treatment that CEA has enjoyed. Thus an ISPOR Task Force Report on the topic is a welcome addition [3].
As the Report argues, a comprehensive assessment of a new health-care intervention at time of launch can benefit from both CEA and BIA. At least in some respects, the two approaches can be viewed as complementary. CEAs evaluate costs and outcomes associated with health-care strategies and can be used to guide coverage or reimbursement decisions. BIAs are used, as the Report notes, for budget planning, for forecasting, and for computing the impact of a health technology on health insurance premiums. BIAs can be helpful in estimating the financial consequences of adoption and diffusion and in predicting how a change in mix of drugs versus other therapies will impact the trajectory of health spending on a condition.
The publication of this Task Force Report will help legitimize the practice. The Task Force has performed its job well and delivered a balanced and thoughtful statement. The call for standards and rigor should help strengthen the field.
Importantly, there is a central focus on the practical needs of the decision-maker, which sets it apart from most reports on CEAs. The recommended perspective is that of the budget holder. The Report notes that the drawing of budget boundaries is a highly local exercise, and that the specific set of assumptions of interest should reflect the decision-maker rather than a “reference case.” The Task Force highlights the fact that analyses should include data that reflect the size, characteristics, and symptoms of the population at hand, as well as the current and new treatment mix. The Report emphasizes that where possible BIAs should incorporate decision-makers’ own populations for other parameter estimates.
There are some important qualifiers here as well, for example, that the purpose of BIAs is not to produce exact estimates but to provide a valid computing framework on possible budget consequences. The Report also usefully recommends “that the analyst use the simplest design that will generate credible and transparent estimates,” a kind of Occam’s razor for the field. There is also useful guidance for the reporting of analyses.
One only wishes the Report had gone further in a few respects. Additional discussion about how BIA are being used—and should be used—would have been helpful. One would have appreciated a worked example, especially for BIAs that attempt to measure cost-offsets associated with a new drug, rather than a mere quantification of the financial consequences of uptake. In fairness, this would have been a considerable undertaking and probably beyond the scope of any task force purview, but this kind of illustrative example was used to great effect by the Panel on Cost-Effectiveness and is something for ISPOR to consider for a future project.
One also would have liked more support for the basic assertion that BIAs is an “essential” part of a comprehensive economic assessment of medical technology. The Report notes that a technology could be efficient but not affordable, and that “in such instances, there is, unfortunately, no current scientific guidance on how to resolve this dilemma.” More commentary on this fundamental question, as well as some illustrative examples and discussion of a future research agenda would have been helpful. Also, the Report emphasizes that models must meet the needs of decision-makers but this begs many questions about what decision-makers really want and how anyone is to know.
A final note: it did occur to this reviewer that budget impact analyses have the unfortunate acronym “BIAS.” One assumes that that is an unhappy coincidence but it does provoke a point: there is concern as always about hidden and optimistic assumptions. The larger wonder, however, is why the drug industry is asked to do the work that health plans should be doing themselves.
These remarks should not detract from the good work of the group. ISPOR Task Force reports have earned a place as well-cited products that help codify and clarify important issues for the field, from the role of modeling to the use of economic evaluation alongside randomized clinical trials [4,5]. They serve as a focal point for reference and further debate. The Task Force Report on Budget Impact Analysis clearly follows in this tradition.
References
1 Gold MR, Siegel JE, Russell LB, Weinstein MC. Cost-Effectiveness in Health and Medicine. Oxford: Oxford University Press, 1996.
2 Drummond MF, Sculpher MJ, Torrance GW, et al. Methods for the Economic Evaluation of Health Care Programmes (3rd ed.). Oxford: Oxford University Press, 2005.
3 Mauskopf JA, Sullivan SD, Annemans L, et al. Principles of good practice for budget impact analysis: report of the ISPOR task force on good research practice—budget impact analysis. Value Health 2007;DOI: 10.1111/j.1524-4733.2007.00187.x.
4 Weinstein MC, O’Brien B, Hornberger H, et al. Principles of good practice for decision analytic modeling in health-care evaluation: report of the ISPOR task force on good research practices—modeling studies. Value Health 2003;6:9–17.
5 Ramsey S, Willke R, Briggs A, et al. Good research practices for cost-effectiveness analysis alongside clinical trials: the ISPOR RCT-CEA task force report. Value Health 2005;8:521–33.
Authors
Peter J. Neumann