ISSUE: Continuous Quality Improvement of Health Care Economic Evaluation Research and Use in Health Care Policy  
 

THE ISSUE:

  • While the average health care researcher or practitioner may be able to follow major health care economic indicators and the release of major health care economic studies, a process to track overall quality of health care economic evaluations and their use in making health care policies is needed.
  • As an example of broad quality monitoring, in 2003, the US Agency for Healthcare Research and Quality (AHRQ) announced it will produce annual reports on national trends in the quality of health care delivery in the United States. AHRQ commissioned the Institute of Medicine to help develop a vision for this report that will allow national and state policy makers, providers, consumers, and the public at large to track trends in health care quality. Envisioning this National Health Care Quality Report offers a framework for measuring health care quality, specific examples of the types of measures that should be included in the report, suggestions on the criteria for selecting measures, as well as advice on reaching the intended audiences.
  • Reviews and periodic quality reports advance international health care efficiency and quality and become a mainstay of our global effort to improve the economics and quality of all health care sectors. Health care economic evaluation quality reports could quantify the adequacy in the quality of health care economic studies.
  • It is important to monitor the quality of health care economic evaluation guidelines being used by journals, organizations and governments around the world for assessing new therapies and allocation of resources.
  • Methods and statistics used in health care economic evaluations need to be critiqued periodically to improve the quality and usefulness for these studies and guidelines.

RECOMMENDED ISPOR ACTIONS:

Form an ISPOR Task Force on quality assessment of health care economic evaluation research and the use of this research in health care policy making. This Task Force will (through surveys, white papers, and sessions at ISPOR meetings):

  • Develop periodic reports on the state and quality of health care economic evaluations and use of these studies in making health care policy,
  • Develop recognition of good health care economic research,
  • Develop a repository of ‘reference cases’ for health care economic research,
  • Develop a report card on the quality of journal guidelines on health care economic evaluations as well as government guidelines on health care economic evaluation information

REFERENCES

Reed SD et al. Conducting economic evaluations alongside multinational clinical trials: Toward a research consensus. American Heart Journal. March 2005;149:434-43.

Neumann PJ, Stone PW, Chapman RH, Sandberg EA, Bell CM. The Quality of Reporting in Published Economic-Utility Analyses, 1976-1997. Annals of Internal Medicine. 2000; 132(12):964-72.

Neumann PJ, Greenberg D, Olchanski NV, Stone PW, Rosen AB. Growth and quality of the economic- utility literature, 1976-2001. Value in Health. 2005;8(1):3-9.

Rosen AB, Greenberg D, Stone PW, Olchanski NV, Neumann PJ. Quality of Abstracts of Papers Reporting
Original Economic-Effectiveness Analyses. Medical Decision Making. In press.

Neumann PJ, Greenberg D, Olchanski NV, Stone PW, Rosen AB. Growth and quality of the economic- utility literature, 1976-2001. Value in Health. 2005;8(1):3-9.

Neumann PJ, Stone PW, Chapman RH, Sandberg EA, Bell CM. The Quality of Reporting in Published Economic-Utility Analyses, 1976-1997. Annals of Internal Medicine. 2000; 132(12):964-72.

Rosen AB, Greenberg D, Olchanski NV, Chapman RH, Neumann PJ. Reporting of Key Data in Abstracts of Economic-Utility Analyses. Abstract SMDM, Chicago, IL, October 19-22, 2003.

APPENDICES: 


Table 1:            Detecting Flaws in Economic Evaluation

Source: Michael Drummond And Mark Sculpher

COMMON METHODOLOGICAL FLAWS IN ECONOMIC EVALUATIONS,
Centre for Health Economics

University of York, UK (prepared for a potential supplement to Medical Care) 

Potential Flaw

Questions for Users to Ask

Omission of important costs or benefits

Given the chosen study perspective, would inclusion of any of the omitted costs or benefits have a big impact on study results?

Selection of an inappropriate alternative for comparison

Are any viable alternatives omitted?

Is the alternative selected for comparison an inefficient treatment option?

Biases in synthesising clinical data

Where effectiveness estimates are based on synthesis (e.g. meta analysis), have potential differences in the clinical studies been recognised and adjusted for?

Inadequate representation of the effectiveness data

Are all the available clinical studies used as a basis for the cost effectiveness study?

If some studies are excluded is this decision justified?

Inappropriate extrapolation beyond the period observed in clinical studies?

Is the time horizon for the economic study adequately justified?

In any extrapolation, are both optimistic and pessimistic scenarios explored?

Excessive use of assumptions rather than data

Are any data available to substantiate the major assumptions in the study?

Is the impact of major assumptions explored through sensitivity analysis?

Inadequate characterisation of uncertainty

Is a confidence interval, or plausible range, presented for the main study parameters?

Does the analyst go beyond simple, one-way sensitivity analysis?

Inappropriate aggregation of results

Are the component parts of the cost effectiveness ratio, or net benefit estimate, presented?

Reporting average cost effectiveness ratios

Are all the comparisons between the alternatives expressed in incremental form?

Lack of consideration of generalisability issues

Does the analyst assume that the study results apply in other settings?

What are the main differences, between the study setting and others, that would affect the cost effectiveness results?

Selective reporting and general emphasis on findings

Are words like ‘substantial’ and ‘minor’ added as descriptors?

Is the discussion of study results even-handed, or is undue emphasis placed on the more positive results?


 

Source: P. Neumann, N..V. Olchanski; A.B. Rosen; D. Greenberg; R. Chapman; P.W. Stone; J. Nadai. ARE PUBLISHED COST-UTILITY ANALYSES IMPROVING? (Poster) ISPOR: Arlington, VA, May 18-21, 2003.

  

 

Figure 1: Changes in CEA Report Quality over Time


Health Science Council Main Page