Standardization

Comparability is the essence that determines the preference of one intervention among alternatives; differences in cost-effectiveness should reflect true differences and not unnecessary differences in method. This panel is not the first to recognize the need for consensus on a set of standards that will promote comparability of studies.

When resources are limited, how are they allo-cated to programs important to the respective seg-ments of society? The Panel on Cost-effectiveness in Health and Medicine [4] recommended cost-effectiveness analysis (CEA) from a societal per-spective for policy decision-making on healthcare resource allocation. They recommended a stan-dardized reference case analysis across all CEAs regardless of the intervention or outcome to pro-vide the methodological uniformity that supports comparability.

Besides health, real-world decisions include other considerations such as access to services, helping the most vulnerable, and other values im-pacted by health decisions. Economic assessment is only one of the tools decision-makers must use, and the information it provides must be weighed within the context of these other criteria. Values outside of healthcare, which often influence choices about health services, cannot be quantified in CEA. Cost-benefit analysis (CBA), cost-effectiveness anal-ysis (CEA), and cost-consequence analysis (CCA) are complementary and the use of one does not preclude the use of others in a study. Although quality-adjusted life-years (QALYs) have the ad-vantage that they measure changes in quality as well as quantity of life, as currently defined, they do not reflect perfectly everything about health that matters to people, and perhaps never can.

The Panel on Cost-effectiveness in Health and Medicine [4] made recommendations concerning items of intervention and outcome to be included in the numerator and the items for the denomina-tor for a reference case scenario. Most are based on reasonable facts, but some are arbitrarily cho-sen and recommended for a reference case to maintain consistency across studies. At present, there is still no standard guide to good modeling practices that can be used as a teaching or refer-ence tool. No clear taxonomy of modeling tech-niques has been documented, and there are no standardized presentation formats.

Making Choices

Beyond standardization, in each study a number of choices are made to fit the model to the research question. Wherever choices are made, con-servative values of all parameters should be cho-sen, and the base case should represent the most plausible assumptions.

When deciding on perspective, societal perspec-tive, which includes all relevant cost and outcomes consequences, is preferable. Certain options in de-cision-making will be cost-effective from the soci-etal perspective and not from the patient’s per-spective. Resource allocation decisions are based on cost-effectiveness evaluated at a specific level. Decisions made at a higher level will affect the re-source availability at lower levels. For example, at the societal (governmental) level, policy decisions on resource allocation are made based on the larg-est proportion of the public affected. When indi-vidual perspective is examined, that segment of the population afflicted with a condition evalu-ated as secondary for resource allocation purposes would need to seek resources elsewhere, and their cost-effectiveness model would have to take this into account. Many healthcare providers find the societal perspective irrelevant for their purposes; a great deal of controversy continues concerning the use of a narrower perspective and whether it should only be presented accompanied by the so-cietal perspective.

Choice of the costs of an intervention from the governmental or societal perspective will take into account the actual wholesale price (AWP) or dis-counted wholesale price (DWP). From the patient perspective there is a question of which price to use. Are cost and price the same? Should the ac-tual price paid be used, or should the discounted retail price, the brand-name drug prices, or generic drug prices be used? Each decision should be transparent and based on sound rationale.

Choice of assumptions should be realistic, re-flecting available data. No model perfectly repre-sents reality; its validity rests on whether its as-sumptions are reasonable in light of the needs and purposes of the decision-maker and whether after close examination its implications make sense. In making discounting decisions, both costs and ben-efits should be done at the same rate, a standard of care should be used as the appropriate compar-ator, and the time horizon should be the duration of time a drug can be expected to meaningfully impact the patient’s health.

Methodological Development

A limitation of decision-tree models is that they are not well suited to represent recurrent events over time [5,6]. In chronic diseases, outcome events such as complications of the disease or its treatment, recurrence of disease, and mortality, are confounded frequently during a lifetime, with probabilities that change with time, age, and health status. Rather than model each event as a separate branch of a complex decision tree, health economic modeling methodology has room for maturation and refinement to allow more efficient mathematical representations of such events. Cur-rent alternatives in development include state-transition models, difference equations, determin-istic models, and stochastic models, or discrete event simulations [7].

Extending Clinical Studies and Data Issues

Some degree of modeling is usually necessary to as-sess clinical outcomes and economic consequences beyond the necessarily limited parameters of a clini-cal trial, and modeling represents the only appropri-ate analytic approach to estimate healthcare utiliza-tion, practice patterns, and other costs associated with observations across defined geographic areas or treatment settings, such as from country to country, health management organization to fee-for-service, or government to private [8].

Many cost and outcomes distributions violate standard normality assumptions, and outliers can have a major impact on results. There are substan-tial problems with aggregation bias when costs and outcomes are averaged or combined for large groups such as disease related group (DRG) reim-bursement levels or average length of stay. As far as possible, data should be analyzed at the individ-ual level for both costs and outcomes. Many still question whether RCT data should be made avail-able to support individual-level analyses.

Analysis of data from all study subjects is nec-essary to support interpretation of clinical trial data for pharmacoeconomic modeling. However, although intent-to-treat analysis is important, it is not necessarily the only way to analyze RCT data for modeling.

Effectiveness Measures

Several other issues arise in the estimation of effec-tiveness for modeling methodology: specification of survival parameters; use of disease-specific or total mortality data; modeling patient characteris-tics; using models to vary program parameters; use of modeling to address lead-time and length biases; estimating uncertainties. The techniques that exist to deal with these issues are serviceable, but have not yet achieved state-of-the-art status or standardization.

Parameter uncertainty is generally handled on a qualitative basis with either univariate or multi-variate sensitivity analysis or max-min analysis, or quantified using statistical approaches such as the Delta method, joint confidence intervals, boot-strapped estimates, or Monte Carlo simulation. No proven method exists to validate structure un-certainty in a model due to either the parameter values assigned or to the mathematical form in which the parameter values are combined, except to compute C/E ratio estimates for each alterna-tive structural assumption and examine appropri-ateness of the results. Even process uncertainty is an unknown. Would any two analysts follow the same model, or if the same problem were posed to an analyst a second time (without awareness of the first result), would the same model be fol-lowed?

While it is generally agreed that proper applica-tion of multivariate sensitivity analysis is necessary, there is ongoing controversy over its value.

Model Validation

As a mathematical device, and as a potentially im-portant component of healthcare decision-mak-ing, credibility of a pharmacoeconomic model rests on its validity. Besides an estimate of the range of uncertainty of its parameters, each model should be shown to demonstrate face validity and predictive validity. Wherever possible, models should be validated against other data sets.

Peer Review of Models

To ensure the quality and enhance the acceptabil-ity of pharmacoeconomic modeling, all models should undergo systematic peer review before pre-sentation. This could be a standardized audit of the structure, process, and validity of the model and would ensure that all salient model results are transparent. A technical peer review would neces-sitate passing an electronic copy of the computer model to the reviewer(s), which raises questions about the handling of proprietary property and confidentiality.

Recommendations and Next Steps

The following recommendations address the seven issues identified:

1. Working towards general acceptance that mod-eling of both costs and effectiveness is a valid and often essential method to inform health-care decision-making will be necessary to establish modeling as an invaluable healthcare decision-making tool.
2. Because the usefulness of modeling studies is necessarily based on comparability, it is impor-tant to assemble a consensus of opinion on standardized practices and policies.
3. Once standardization has been achieved, a ref-erence text of these practices should be pre-pared and disseminated.
4. Pharmacoeconomic claims based on these gen-erally accepted modeling approaches should be permitted by regulatory agencies, and should always include transparency and appropriate disclaimers such as: “This economic analysis is based on assumptions and simulations con-cerning the efficacy of [drug name] that meet FDA criteria for claims of efficacy.” Any model that relies on assumptions about a drug’s efficacy that are not based on data from RCTs must prominently disclose such limita-tion in any promotion.
5. We recommend that as an independent profes-sional association of pharmacoeconomic and outcomes researchers, ISPOR take the initia-tive of assembling a balanced international panel of thought-leaders and end-users in the field of modeling to develop a package of gen-erally accepted modeling practices, building upon previously published work.
6. Once these practices have been documented, the goal of ISPOR should be to encourage all stakeholders (professional societies, manufac-turing associations, journals, government agen-cies, regulatory agencies, payers, and healthcare providers) to accept these as standards and to endorse their use