|
ISSUE: Continuous Quality
Improvement of Health Care Economic Evaluation Research and Use
in Health Care Policy
THE ISSUE:
- While the average health care researcher or practitioner
may be able to follow major health care economic indicators
and the release of major health care economic studies, a
process to track overall quality of health care economic
evaluations and their use in making health care policies is
needed.
- As an example of broad quality monitoring, in 2003, the US
Agency for Healthcare Research and Quality (AHRQ) announced it
will produce annual reports on national trends in the quality of
health care delivery in the United States. AHRQ commissioned the
Institute of Medicine to help develop a vision for this report
that will allow national and state policy makers, providers,
consumers, and the public at large to track trends in health
care quality. Envisioning this National Health Care Quality
Report offers a framework for measuring health care quality,
specific examples of the types of measures that should be
included in the report, suggestions on the criteria for
selecting measures, as well as advice on reaching the intended
audiences.
- Reviews and periodic quality reports advance international
health care efficiency and quality and become a mainstay of our
global effort to improve the economics and quality of
all health care sectors. Health care economic evaluation quality
reports could quantify the adequacy in the quality of health
care economic studies.
- It is important to monitor the quality of health care
economic evaluation guidelines being used by journals,
organizations and governments around the world for assessing
new therapies and allocation of resources.
- Methods and statistics used in health care economic
evaluations need to be critiqued periodically to improve the
quality and usefulness for these studies and guidelines.
RECOMMENDED ISPOR ACTIONS:
Form an ISPOR Task Force on quality assessment of health care
economic evaluation research and the use of this research in
health care policy making. This Task Force will (through
surveys, white papers, and sessions at ISPOR meetings):
- Develop periodic reports on the state and quality of
health care economic evaluations and use of these studies in
making health care policy,
- Develop recognition of good health care economic research,
- Develop a repository of ‘reference cases’ for health care
economic research,
- Develop a report card on the quality of journal guidelines
on health care economic evaluations as well as government
guidelines on health care economic evaluation information
REFERENCES
Reed SD et al. Conducting economic evaluations alongside
multinational clinical trials: Toward a research consensus.
American Heart Journal. March 2005;149:434-43.
Neumann PJ, Stone PW, Chapman RH, Sandberg EA, Bell CM. The
Quality of Reporting in Published Economic-Utility Analyses,
1976-1997. Annals of Internal Medicine. 2000; 132(12):964-72.
Neumann PJ, Greenberg D, Olchanski NV, Stone PW, Rosen AB.
Growth and quality of the economic- utility literature,
1976-2001. Value in Health. 2005;8(1):3-9.
Rosen AB, Greenberg D, Stone PW, Olchanski NV, Neumann PJ.
Quality of Abstracts of Papers Reporting
Original Economic-Effectiveness Analyses. Medical Decision
Making. In press.
Neumann PJ, Greenberg D, Olchanski NV, Stone PW, Rosen AB.
Growth and quality of the economic- utility literature,
1976-2001. Value in Health. 2005;8(1):3-9.
Neumann PJ, Stone PW, Chapman RH, Sandberg EA, Bell CM. The
Quality of Reporting in Published Economic-Utility Analyses,
1976-1997. Annals of Internal Medicine. 2000; 132(12):964-72.
Rosen AB, Greenberg D, Olchanski NV, Chapman RH, Neumann PJ.
Reporting of Key Data in Abstracts of Economic-Utility Analyses.
Abstract SMDM, Chicago, IL, October 19-22, 2003.
APPENDICES:
Table 1:
Detecting Flaws in Economic Evaluation |
|
Source: Michael Drummond And
Mark Sculpher
COMMON METHODOLOGICAL FLAWS IN ECONOMIC EVALUATIONS,
Centre
for Health Economics
University of York, UK (prepared
for a potential supplement to Medical Care)
|
Potential
Flaw
|
Questions
for Users to Ask
|
|
Omission of important costs or
benefits |
Given the chosen study
perspective, would inclusion of any of the omitted costs or
benefits have a big impact on study results? |
|
Selection of an inappropriate
alternative for comparison |
Are any viable alternatives
omitted?
Is the alternative selected for
comparison an inefficient treatment option? |
|
Biases in synthesising clinical
data |
|
|
Inadequate representation of the
effectiveness data |
Are all the available clinical
studies used as a basis for the cost effectiveness study?
If some studies are excluded is
this decision justified? |
|
Inappropriate extrapolation
beyond the period observed in clinical studies? |
Is the time horizon for the
economic study adequately justified?
In any extrapolation, are both
optimistic and pessimistic scenarios explored? |
|
Excessive use of assumptions
rather than data |
Are any data available to
substantiate the major assumptions in the study?
Is the impact of major
assumptions explored through sensitivity analysis? |
|
Inadequate characterisation of
uncertainty |
Is a confidence interval, or
plausible range, presented for the main study parameters?
Does the analyst go beyond
simple, one-way sensitivity analysis? |
|
Inappropriate aggregation of
results |
Are the component parts of the
cost effectiveness ratio, or net benefit estimate,
presented? |
|
Reporting average cost
effectiveness ratios |
Are all the comparisons between
the alternatives expressed in incremental form? |
|
Lack of consideration of
generalisability issues |
Does the analyst assume
that the study results apply in other settings?
What are the main differences,
between the study setting and others, that would affect the
cost effectiveness results? |
|
Selective reporting and general
emphasis on findings |
Are words like ‘substantial’ and
‘minor’ added as descriptors?
Is the discussion of study
results even-handed, or is undue emphasis placed on the more
positive results? |
|
Source: P. Neumann, N..V.
Olchanski; A.B. Rosen; D. Greenberg; R. Chapman; P.W.
Stone; J. Nadai. ARE PUBLISHED COST-UTILITY ANALYSES
IMPROVING? (Poster) ISPOR: Arlington, VA, May 18-21, 2003. |
Figure 1: Changes in CEA Report Quality over Time
|
