Health care administrators, policy makers, and practitioners must balance the needs and desires of individual patients with the needs and desires of society at large, realizing that not all needs and desires can be met. Information comparing the expected gains of a medical intervention against the expected cost of that intervention versus other health care interventions are, many times, difficult to interpret or compare. The mission of the International Society for Pharmacoeconomics and Outcomes Research is to translate pharmacoeconomics and outcomes research into practice to ensure that society allocates scarce health care resources wisely, fairly, and efficiently. Toward this mission, ISPOR, co-sponsored with the U.S. Health and Human Services, Agency for Health Care Policy and Research, and the Health Outcomes Work Group of the Pharmaceutical Research and Manufacturers of America, convened an Advisory Panel Meeting and Conference on Pharmacoeconomic Issues, February, 1998. This conference provided a forum for researchers and practitioners to communicate needs and concerns as consensus is developed on methodology, interpretation, and use of pharmacoeconomic information.

The objective of this interdisciplinary conference was to identify the issues in conducting pharmacoeconomic studies, interpreting the results of these studies, and using pharmacoeconomic information in health care decisions. The specific goals of the conference were to:

1. identify key contentious methodology issues in conducting health care economic evaluations with clinical studies
2. identify key contentious methodology issues in conducting health care economic evaluations using modeling studies
3. identify key contentious methodology issues in conducting health care economic evaluations using databases
4. determine the education and skills needed for conducting pharmacoeconomic evaluations in health care decisions
5. identify the issues in application of economic evaluations in health care intervention protocol development, formulary decisions, and practice guideline development and use
6. identify the issues in addressing bias, credibility, and quality of pharmacoeconomic evaluations
7. identify the issues in communicating and reporting health care economic evaluation information

During this conference, sixty-one pharmacoeconomics and outcomes researchers, clinical practitioners and healthcare decision-makers in the United States met to develop consensus on issues relating to pharmacoeconomic and outcomes research evaluations and the use of these evaluations in healthcare decisions. The results of this deliberation are as follows:

METHODOLOGICAL ISSUES IN CONDUCTING PHARMACOECONOMIC EVALUATION - CLINICAL STUDIES

Pharmacoeconomic methods used to assess cost alone or other measures of value often fall short of regulatory standards. Conversely, study methods used to demonstrate drug efficacy, such as randomized Clinical studies, are insufficient for addressing the question of value in other applied settings. To overcome the limitations of using clinical studies data for health economic evaluations, researchers, decision-makers, policy-makers and consumers should be well-versed in the appropriate use of clinical studies

There are four key issues are as follows:

1. When should randomized clinical studies be the primary approach to assessing questions of value?
2. What modifications to randomized clinical studies would improve their usefulness as tools for health economic decision-making?
3. When should observational studies be used to assess questions of value?
4. What modifications to observational studies would improve their usefulness as tools for health economic decision-making?

The recommendations for these issues are:

1. Define' usual care': As trials adopt more naturalistic designs, the use of "usual care" as comparator will likely increase and comparability will be necessary.
2. Develop new methods to account for protocol-related costs
3. Develop alternative methods for intent-to-treat analyses in usual care trials Although the intent-to-treat analysis is important, there may be other research questions important to decision-makers that would benefit from alternate or additional analysis.
4. Address problems of pooling economic data due to significant differences in clinical practice across settings and sites, especially in international studies.
5. Establish the range for "acceptable" levels of certainty of study results to inform value-based decision-making.
6. Improve statistical methods for adjusting for selection bias, for example, instrumental variables and propensity scores.
7. Develop better methods to estimate variance around the components of value, such as bootstrapping and re-sampling techniques.
8. Develop systematic comparison of randomized Clinical studies and observational studies on the same interventions.
9. Survey other disciplines (such as psychology, sociology, marketing research) for new approaches to methodology, particularly in the areas of data collection, analyses and instrumentation.
10. Use large, simple trials to measure resource utilization such as early Phase IV studies,.
11. Develop better methods to measure direct medical costs not routinely captured, such as nursing time and telephone care (OMERACT panel).
12. Develop better methods to measure relevant indirect costs such as caregiver time or lost productivity (ARAMIS). As electronic medical records evolve and expand, encourage inclusion of standardized outcome measures.

METHODOLOGICAL ISSUES IN CONDUCTING PHARMACOECONOMIC EVALUATION - MODELING STUDIES

The primary purpose of modeling is to inform the decision-making process. One considerable benefit of model formalization is that the uncertainties and assumptions in this process are made explicit and transparent. Currently there are two major obstacles confronting modeling methodology currently: a) how to optimize the production of useful information for health economic decision-makers and b) how to encourage its acceptance and use of information

There are seven key issues (areas of controversy) in modeling methodology as follows:

1. Standardization
2. Making choices
3. Methodological development
4. Extending Clinical studies and data issues
5. Effectiveness measures
6. Model validation
7. Peer review.

The recommendations for these issues are:

1. Work towards general acceptance that modeling of both costs and effectiveness as a valid and often essential method to inform health care decision-making
2. Assemble a consensus of opinion on standardized practices and policies
3. Prepare and disseminate a reference text of these practices once standardization has been achieved.
4. Permit pharmacoeconomic claims based on these generally accepted modeling approaches by regulatory agencies, and should always include transparency and appropriate disclaimers such as: "This economic analysis is based on assumptions and simulations concerning the efficacy of [drug name] that meet FDA criteria for claims of efficacy." Any model that relies on assumptions about a drug's efficacy that are not based on data from RCTs must prominently disclose such limitation in any promotion.
5. Initiate and assemble a balanced international panel of thought-leaders and end-users in the field of modeling to develop a package of generally accepted modeling practices
6. Encourage all stakeholders, professional societies, manufacturing associations, journals, government agencies, regulatory agencies, payers and health care providers, to accept these as standards and to endorse their use once these practices have been documented.

METHODOLOGICAL ISSUES IN CONDUCTING PHARMACOECONOMIC EVALUATIONS - RETROSPECTIVE AND CLAIMS DATA STUDIES

Health care decision-makers require rapid access to information. The evidence that assists decision-makers to draw conclusions often has not been available. Both RCTs, and retrospective methods using existing databases, provide such information, and typically answer different questions. Most RCTs are designed to measure efficacy, not effectiveness. "Real world" data can be provided by database studies.

Eight key issues were identified as follows:

1. What research questions can be answered by retrospective analyses?
2. What data sources are available to answer these questions?
3. How is cost-effectiveness measured using automated databases?
4. How can data quality within a database be evaluated?
5. What types of statistical methods can be utilized to control for treatment effects?
6. What potential types of bias exist in retrospective database analyses?
7. What alternative methods for assessing selection bias are available?
8. How can transparency be ensured in retrospective database analyses?

The recommendations for these issues areas follows:

1. Begin retrospective database analysis studies with a clear question and design, based on guidelines for good epidemiological practices.
2. Ensure privacy of individuals at all times in retrospective database analyses.
3. Use techniques that exist to address shortcomings of retrospective data sets
4. Subject multivariate models to extensive specification testing.
5. Examin age- or gender-adjusted utilization rates and annual per capita expense by payer, health plan, geographic region and country
6. Augment administrative databases, frequently used for retrospective pharmacoeconomic studies, to include more clinical information
7. Establish standard measures to deal with all areas of potential bias.

EDUCATION AND SKILLS NEEDED TO CONDUCT, INTERPRET AND USE ECONOMIC EVALUATIONS IN HEALTHCARE

Like other disciplines, to expand and grow as a mature area of research and application, the field of health economics requires experts and skilled professionals. Unlike many other scientific fields, there is no one background or training that prepares the researcher or the user of health economic information, who currently come from a diversity of educational and experiential backgrounds

The key issues related to education and skills in the field of health economics are as follows:

1. Multidisciplinary Programs - The structuring of multidisciplinary programs needs to be defined for people coming from a variety of backgrounds.
2. "Real-World" Applications - Training must include "real-world" applications.
3. Ideal Program: It is unlikely that an ideal program can be created in any one place within one institution or group without collaboration with others.
4. Minimal Competencies - The usefulness of minimal competencies in the field has to be determined. Minimal competencies will be different for current and future practitioners by depth of involvement. For each level of involvement, the type of competencies will have to be defined.
5. Who should be trained - Questions pertaining to who should be trained, how should training be performed and what level of training is required need to be answered.
6. Credentialing - The necessity of credentialing needs to be assessed.
7. Training Opportunities - There is need to improve the way information about training opportunities is disseminated.

The recommendations for these issues are as follows:

1. Develop three levels of expertise: awareness, application, and conceptualization
2. Develop access to detailed information about available educational programs in the field of health economics
3. Utilize relevant educational resources outside of health economics to enhance the educational infrastructure
4. Accommodate multidisciplinary participants through the availability of prerequisite trainings and flexible core course offerings for degree programs
5. Balance didactic and experiential education
6. Develop a credentialing process to establish standards for the field
7. Standardize training and certification through a three step process: a) develop guidelines for post-professional degree training; b) accredit pharmacoeconomic residencies and fellowships; c) establish collaborations with other organizations to expand accreditation to other relevant residencies

APPLICATION OF HEALTHCARE INTERVENTION ECONOMIC EVALUATIONS IN HEALTHCARE DECISION-MAKING

Information about the impact of new therapies on costs within a healthcare system should be essential for making better healthcare decisions. However, the relevance of health economic information to decision-makers has not been demonstrated. There is little user-friendliness in the health economic data. There is a lack of consistency of approach and format that would facilitate comparison under review. Finally, much of the information presented lacks the transparency necessary for the user to determine the appropriateness of methods or the soundness of assumptions. A fundamental disconnect exists between a) the way decisions are made by healthcare decision makers, b) type of information presented to healthcare decision makers and c) the type of information required by health economic and outcomes researchers.

There are eleven key issues as follows:

1. Evaluative criteria are often weighed differently by potential users of health economic research data for decision-making purposes such as formulary committees, providers, health plan managers, patients or employers.
2. Language and definitional barriers hinder effective communication between potential users and producers of the information may exist.
3. Lack of treatment comparisons. There is little data on direct product or treatment comparisons that are of greatest interest to potential users.
4. Lack of communication. Little dialogue exists between the potential users and the producers of information on what is relevant and what information can be generated by health economic research.
5. Lack of understanding. Potential users of health economic research data may be hesitant to include health economic information in their decision-making process because it is different from their established clinical orientation.
6. Conflicting study design. To conduct studies which provide health economic information that meets users' needs, certain research designs for health economic and outcomes research studies may conflict with clinical design, causing regulatory and liability concerns.
7. Research Funding. Some sources of research funding may present a barrier to the credibility and application of study results.
8. Societal perspective vs. individual perspective. A conflict may exist between recommendations based on population data and the care of individual patients.
9. Lack of quality criteria. When health economic research data is used in the decision-making process, there is no recognized approach for measuring the quality of the decision or the net result.
10. Lack of experts. There are few skilled opinion leaders or other resources from which potential users can seek advice and assistance.
11. Segregation of organizational finances and health outcome decisions. Decision-maker organizations segregate budgetary decisions for pharmaceuticals from those related to other medical technologies and services.

The recommendations for these issues are:

1. Recruit a central organizer to coordinate the improvement of the application of healthcare intervention economic evaluations in healthcare decision-making.
2. Create focus groups to provide a forum for dialogue between potential users, producers, and regulators of information Researchers and suppliers of health economic data must actively engage with decision-makers to determine the key health economic evaluation criteria for decision-making purposes and formulate ways to supply the information consistently.
3. Determine a set of variables which researchers can supply. Decisions are seldom made using a single variable.
4. Develops a set of simple criteria for evaluation of these studies, agreed upon by consensus of all parties involved and designed to recognize different types of perspectives and research design so that specified research questions and business needs are met.
5. Seek to bolster the objectivity, reliability, and credibility of the health economic studies through various mechanisms, including working with sponsors, researchers and journal editors, to adopt protocols that will establish the independence of research and statements for the disclosure of funding sources.
6. Offer training for decision-makers in using health economic research information for decision-making. A consortium of managed care and other purchasing organizations, academic researchers and one or more health economic research organizations should be formed to execute this recommendation.
7. Develop a standard reporting format to allow flexible weighting of factors based on individual decision-making preferences. The presentation of the results of health economic analyses be modified to show the various components of effectiveness measures, service utilization measures and costs.
8. Form a committee that would produce a standard format for Data Element Shells (DES). DES would be in the form of desegregated data from cost-consequence or cost-effectiveness analyses. For each outcome of interest, reference to the data source could be made to allow reviewers to discern the degree of scientific support for each data element. ISPOR, in collaboration with potential users and producers of information, could be responsible for creation and updating of a DES form. The ISPOR committee would decide on the level of specificity of the DES, perhaps either a general format for all drugs or a specific format for individual drug classes.
9. Support an information clearinghouse of available thought-leaders and experts in the field. This should include development and maintenance of an Internet WEB site with links to expert's homepages and e-mail addresses. ISPOR as an organization brings together many of the researchers qualified to evaluate health economic research and interpret findings.
10. Develop rosters of persons qualified to review studies, similar to editorial boards for journals, where the reviewers would agree to participate in reviewing documents or addressing queries to promote a better understanding of the field of health care economics.

ADDRESSING QUESTIONS OF BIAS, CREDIBILITY AND QUALITY IN HEALTH ECONOMIC EVALUATIONS

Multiple published studies have criticized the rigor, relevance, objectivity, methods, and reports produced within the health economic research domain. Consequently, health economic research findings are not used as extensively as they could be and rational decision processes about the efficient use of health care resources may not be fully informed. Ultimately, care for patients and populations may be adversely affected. In this context, there is a need for continued improvement in the quality of economic research conducted.

There are three key issues as follows:

1. Quality: are best methods being used?
2. Bias: whether real or perceived, how do we deal with it?
3. Credibility: do we have a problem with believability or with relevance?

The recommendations are as follows:

1. Design and conduct studies using the best available practices consistent with the study objectives
2. Disclose any financial relationships which authors and speakers have linking them directly or indirectly to the interventions under study
3. Authorship should conform to generally recognized practices among the peer research community. Research data, given full disclosure, transparency, and sufficient information to replicate the study, should be judged on the merits of its content
4. Develop a code of ethics for health economic researchers
5. Develop study methodology practice standards
6. Convene a conference similar to this conference in 2 years to evaluate progress and recommend next steps

COMMUNICATION AND REPORTING HEALTH ECONOMIC INFORMATION

Users of health economic information represent many different perspectives with various levels of expertise and information needs. To obtain most value from the resources invested in health economic research, how do we optimize the effectiveness of ou communicating of health economic information?

There are three key issues as follows:

1. Relevance: Is it needed?:
2. Usefulness: Will the intended audience be able to make use of it?
3. Credibility: Is it believable?

The recommendations are as follows:

1. Identify the needs of users of health care economic information. A survey of all users of health economic information, will provide a basis for standardization of communications.
2. Establish standard communication formats based on predetermined relevance, information and credibility needs of users and on standard health economic performance standards that should be under development elsewhere. These should eventually include: a) uniform presentation, standard terminology, adequate disclosure; and a basis in previously published guidelines
3. Adopt a Reporting Guidance (RGs) and apply to all publicly presented communications, as standardized formats are established
4. Evaluate the use of Reporting Guidance and the quality of reporting on a biannual basis.
5. Establish a principle of publicly accessible reports that adhere to ISPOR RGs. This would allow access to research reports that is not directly controlled by the researcher or the research organization. Once a report has been "filed" for public accessibility, all subsequent communications could refer to that report
6. Institute an enhanced mode of peer review for all forms of health economic communications. This type of review would assure that there was compliance with ISPOR RGs and fair, full and adequate disclosure, allow for review of the underlying data and any model used, and confirm that all other ISPOR standards for the conduct of health economic studies have been met.

EXPECTED OUTCOMES OF THIS CONFERENCE

The following are the expected outcomes (products) of this conference in addition to this report are:

1. Publication: This report will be published in VALUE IN HEALTH, Journal of the International Society for Pharmacoeconomics and Outcomes Research
2. Follow-up Activities: Conferences are planned based on the recommendations given in this report
3. ISPOR Policy Statements: Specific policies of the International Society for Pharmacoeconomics and Outcomes Research will be developed from these recommendations ISPOR, in cooperation with other scientific and practitioner organizations and institutional organizations, will work to implement these policies. Examples of these organizations are the Health Outcomes Work Group of the Pharmaceutical Research Manufacturers of America and the Pharmaceutical Research Standards Committee of the American Managed Care Pharmacy Association.
4. Agenda for Future Research Activities The recommendations included in these reports are suggested to be the agenda for future research activities by the Agency for Health Care Policy and Research.

Contact ISPOR @ info@ispor.org  |  View Legal Disclaimer
©2008 International Society for Pharmacoeconomics and Outcomes Research. All rights reserved under International and Pan-American Copyright Conventions. 
Website design by Eagle Systems USA, Inc.