Summary Measures of Population Health Task Force

INTRODUCTION
The US has no metrics for tracking or reporting on patient-reported outcome (PRO) measures of health for the population as a whole. That is, national policymakers have no obvious way to summarize the generic health of the US population (or subgroups) through a single number (or at most a few numbers). Hence, no analog exists to the “consumer price index” or similar measures of the health of the economy that could yield a thumbnail view of the health of the nation from the perspective of patients themselves. The nation is thus unable to know, across the population as a whole, how people are faring on health and wellbeing measures that may be of greatest salience to them.

Summary measures of population health (SMPH) can bridge the gap between utilization, expenditure and disease-specific surveys (on the one hand) and general health information at the population level. Such information would markedly improve health policymaking over time. Early in this decade a US Interagency Working Group on Summary Measures of Health (IAWG) proposed the development of a data resource of measured of health status, including summary health measures. (See Appendix A or http://www.cdc.gov/NCHS/otheract/IAWG/StartingPointProposal.pdf) At an ISPOR-sponsored invitational workshop on Summmary Measure of Population Health: A Working Group Meeting¹, Pennifer Erickson stated that, “summary measures of health [are] a combination of person self-report information with mortality or survival information.” At this Working Group Meeting¹, Michael Wolfson stated that:

“The basic vision is a fundamental reorientation away from inputs and throughputs in the health (illness) care system and toward toward a focus on the overall health of the population (monitoring), and the myriad interventions made in the name of improving health (assessing performance).

The analogy is that a summary measure of population health is the equivalent of a ‘gross domestic product’ for health, in which GDP is itself an element of a system of national accounts. Generic health status (i.e., in today’s lexicon, patient-reported outcomes, or PROs) is the metric; dollars (or euros, or pounds, or ...) are not the numeraire. Thus, a SMPH becomes the numeraire for monitoring, causal analysis, and cost-effectiveness analysis throughout any country’s health system.”

BACKGROUND
To judge how US is doing over time in health and wellbeing, we rely largely on mortality and morbidity statistics. These are often single measures (e.g., infant mortality; incidence and prevalence of type 2 diabetes; some disability statistics), but these are “outside the skin” kinds of measures and do not reflect PRO domains well. That is, the country has little that tells us, across all populations, how people actually feel about functioning, emotional health, cognitive functioning, pain, fatigue, and similar domains.

Some national surveys (in the United States, but also in other countries) may come close to providing data that can be aggregated into a summary measure of population health; these may include (for the US) the National Health Interview Survey, the National Health and Nutrition Examination Survey, the Medical Expenditures Panel Survey, and the Medicare Health Outcomes Survey. State-based BRFSS data (based on HRQOL measures) provide some insights, but possibly not as useful over time or nationally generalizable. Other, more specialized sample surveys may also acquire some PRO or HRQOL data as well.

The US thus has only imperfect ways to determine what we are getting from health expenditures, although clearly on the basis of mortality and morbidity measures themselves the US does not do well on international comparisons. Similarly, the US does not do as well as other nations on issues such as perceived satisfaction or quality of care. A recent update of a Commonwealth Fund study² of five countries reports as follows: Compared with five other nations—Australia, Canada, Germany, New Zealand, the United Kingdom—the US health care system ranks last or next-to-last on five dimensions of a high performance health system: quality, access, efficiency, equity, and healthy lives.  We have no obvious way to determine whether, on PRO-type measures, people believe they are getting their money’s worth.

Moreover, we have no easy means of determining whether such outcomes are getting better or worse over time and whether they differ by population subgroups. Both are important policy issues for this country (like others) for which the risk of extraordinary, and uncontrollable, health care costs is very high.

DOMAIN OF THIS ISSUE
This issue seems to lie largely within the universe of outcomes research -- that is, it is related more to PROs (i.e., quality of life, or health-related quality of life) than to purely clinical outcomes (e.g., those defined by clinicians or relying on laboratory measures). It is, indirectly, related to, economic outcomes, insofar as such summary measures might let us evaluate returns on health care spending. Thus, in theory, this issue can also be related to health policymaking, but it is not intended (nor would it be terribly relevant) for health care decisionmaking at the individual patient level.

The issue also has substantial methodology aspects, given questions such as whether to use indexes or profile measures and how to build in death (survival) as part of the measurement scheme. In addition, the advent of the NIH PROMIS initiative (Patient-Reported Outcomes Measurement Information System) is a major contributor to advances in PRO assessment, and it offers a web-based and computer-adaptive testing (CAT) capability for PRO measurement nationally that was not available as recently as 5 years ago.

RECOMMENDED ISPOR ACTIONS

1. Review the “state of the science” of summary measures of population health and prepare an inventory of current health interview surveys or similar sources of information on PROs at the population or subpopulation level. For policy purposes, subpopulation levels might be defined more by sociodemographics than by disease state. The scope should be as international as possible within constraints of time and resources.
2. Determine whether a knowledge gap exists in this area in the United States and, if so, whether ISPOR efforts to promote the development and use of summary measures of patient-reported population health (i.e., health-related quality of life or PRO measures, presumably also taking death/survival data into account) and determine whether and how, in future, countries could be compared on such summary measures.
3. Include an international component, partly to determine what exemplars exist elsewhere (e.g., in Canada, in the European Union as a whole) and how US-specific information might be compared and contrasted with that from other relevant nations

REFERENCES

1. Summary Measures of Population Health: A Working Group Meeting, sponsored by ISPOR, May 17, 2003, Arlington, VA, USA.
2. K. Davis, C. Schoen, S. C. Schoenbaum, M. M. Doty, A. L. Holmgren, J. L. Kriss, and K. K. Shea, Mirror, Mirror on the Wall: An International Update on the Comparative Performance of American Health Care, The Commonwealth Fund, May 2007. http://www.commonwealthfund.org/publications/publications_show.htm?doc_id=482678&#areaCitation.

APPENDIX A

Interagency Working Group on Summary Measures of Health A "Starting Point" Proposal

The Interagency Working Group on Summary Measures of Health (IAWG), chaired by NCHS Director Edward Sondik, and composed of staff from various DHHS agencies, proposes the development of a data resource of measures of health status, including summary health measures. The IAWG recognizes that the measurement of health is a rapidly advancing field that includes a variety of measurement approaches and involves researchers from a broad range of disciplines. However, it also recognizes that some areas of development of health status assessment and health preference assessment are limited by the lack of large-scale data collection, including different instruments and measurement approaches for a broad range of population measures. These limitations are particularly important for measurement of population health, for translations between different instruments, and for new, data-intensive instrument development and assessment strategies.

promote and support research into health status assessment, health preference assessment, and summary measures of health, which could not occur in the absence of such a resource. Secondarily, to the extent that such a resource measures the health of the population, a goal would be to track population health for use in guiding health policy. The IAWG proposes to build the research on recommendations from an Institute of Medicine sponsored meeting on summary measures of health,¹ as well as on recent recommendations from a peer review of the World Health Organization’s World Health Report 2000.^2,3 These recommendations address continued development of measures as well as the use and “field” testing of several different measures. This work would allow the various underlying assumptions on values, methods, and distributive aspects to be more fully developed. Work with decision-makers (and the public) would also be important to understand how characteristics of measures influence decision-making.

With respect to the public, research is needed on how to examine “public attitudes and reasoning processes related to resource allocations and valued health states¹.” Still additional research would address how best to capture the distributive aspects of health—more specifically the impact on decision-making of public and decision-maker preferences for various distributions of health status and health resources.

It is also important to address the philosophical and theoretical underpinnings of existing measures along with their capability to capture future uncertainties in health states and the critical area of relating current preferences for health states to the likelihood and preferences for future health states.
Along with these fundamental research lines, the IAWG believes it is very important to pursue the more focused goal of comparing the properties and performance of existing measures and instruments in assessing the health status of individuals, the U.S. population, and important sub-populations. Toward this end two specific “tools” may be especially useful.

Comparison of Existing Summary Measures
Important products of the research would be the detailed analyses of the performance of different summary measures in like populations, including head-to-head comparisons of different measures. These analyses, in turn, would facilitate comparisons across studies that use different assessment instruments. Such a data resource will provide, therefore, a critical resource to support future research efforts aimed at developing or refining summary measures of health and health assessment instruments.

Survey of Preferences for Health States
A separable, but important, second aspect of this project will be to develop better information on individuals’ preferences for different health states. Estimates of the preference weights (also called “utility weights”) associated with different health states are essential in constructing summary measures to be used for economic evaluations such as cost-effectiveness analysis. However, the preference weights used in existing summary measures are not necessarily representative of the U.S. resident population. Also, the implications of the methods used to elicit preferences often are not well understood by those who use the instruments to monitor changes in quality of life or by those who interpret the results to guide policy.

The Role Of The Workshop
The role of the workshop is to provide a structured forum for the IAWG to ask for your help. The proposed data resource would be costly to implement, and if longitudinal, to maintain. An extensive inventory of instruments asked of all participants in such a data collection exercise would represent an infeasible respondent burden. Thus, the IAWG has developed this proposal as a starting point to receive feedback to insure such a resource would be developed to most optimally promote future research. The IAWG asks that workshop participants help answer the following questions to further develop this proposal.

1) What are the research priorities in the different areas of measurement of health which could be addressed by a large-scale data collection effort of health status and health preference instruments? The focus on answering this question should pertain to such a data resource. The IAWG is interested in receiving input from a broad range of experts on the priorities identified as well as additional priorities, including potentially:

• Are measures responsive to change over time?
• What is the relationship between health status measures, health preference measures, and clinical and biologic measures; are measures responsive to clinically significant conditions and events?

2) What instruments and assessments should be included in such a data resource? Candidate measures include, although are not necessarily limited to: the Health Utilities Index (Mark 2⁴ and/or Mark 3 ⁵); the Quality of Well-Being Scale ⁶ (QWB); the SF-36 ⁷ (or RAND-36) and its derivatives; the EQ-5D ⁸ instrument developed by the EuroQol group; the Health and Activity Limitation Index ⁹ (HALex); and the World Health Organization=s WHOQOL-BREF ¹⁰ instrument, among other possibilities. The scope and design of the final augmented data set must necessarily balance the goal of facilitating detailed comparisons among many summary measures against the need to limit the burden on survey respondents.

3) What are the additional data needs of such a resource? Essentially, a goal is to find the “minimum data set” with which to foster research into the measurement of health. Thus, research into using novel psychometric techniques to refine current instruments might require further domain-specific questions to refine the responsiveness of current instruments to a broad range of function on a specific domain. Understanding whether an instrument is responsive to the presence or absence of a health condition would require collecting data on respondents’ health histories. What other data are needed to produce an optimal resource, within the constraints of respondent burden?
4) Whom does the IAWG need to survey for this resource? The answer to this question will depend on the perceived research priorities. Measurement of population health would require a nationally representative sample, yet this approach might not yield sufficient numbers to examine in detail the relationship between measures and specific conditions. A detailed comparison of measures against clinical and biologic measures for a specific condition would require sufficient rationale for the need for a large-scale data collection effort as opposed to research through current funding systems. A hybrid approach with a nationally representative sample with over-sampling for priority populations or conditions may be ideal but would need to be considered within the constraints of available survey mechanisms. Ensuring adequate power to answer important research questions will also be a priority.

5) What is the ideal vehicle for data collection for such a resource? Within the framework of issues developed from the prior questions, the IAWG needs to determine the optimal mechanism for collecting the data that will form the resource. While several national surveys, such as the NHIS, NHANES, and MEPS, have been identified, the capacity for additional data collection for these surveys is limited, and the resource would be constrained to their sampling frames. However, development of a new survey mechanism would dramatically increase the resources necessary to implement the resource. Identifying the priorities for research and the necessary populations to survey will assist with identifying the optimal mechanism for collecting the health status data.

¹ Summarizing Population Health. Directions for the Development and Application of Population Metrics. Committee on Summary Measures of Population Health. Marilyn J. Field and Marthe R. Gold, Eds. Institute of Medicine. National Academy Press. Washington, DC. 1998

² World Health Report 2000. Health Systems: Improving Performance. World Health Organization. Geneva, Switzerland. 2000

³ Special Peer Review of the World Health Report 2000

⁴ Torrance GW et al. Multiattribute utility function for a comprehensive health status classification system: Health Utilities Index Mark 2. Medical Care 34(7): 702-722, 1996.

⁵ Torrance GW et al. A multilinear multi-attribute utility function for the Health Utilities Index Mark 3 (HUI3). Medical Decision Making 18(4): 490-, 1998.

⁶ Kaplan RM et al. The Quality of Well-Being scale: Applications in AIDS, cystic fibrosis, and arthritis. Medical Care 27(3): S27-S43, 1989.

⁷ Ware JE and Gandek B. Overview of the SF-36 health survey and the International Quality of Life Assessment (IQOLA) project. Journal of Clinical Epidemiology 51(11): 903-912, 1998.

⁸ The EuroQol Group. EuroQol B a new facility for the measurement of health-related quality of life. Health Policy 16: 199-208, 1990.

⁹ Erickson P. Evaluation of a population-based measure of quality of life: the Health and Activity Limitation Index (HALex). Quality of Life Research 7: 101-114, 1998.

¹⁰ The WHOQOL Group. Development of the World Health Organization WHOQOL-BREF Quality of Life Assessment. Psychological Medicine 28: 551-558, 1998.

¹¹Yen-pin Chiang, COER, ARHQ, Funding Recommendation Memo on AUS Valuation of the EuroQol Group’s EQ-5D.@ Grant No.:R01 HS10243-01A1. P.I.: Stephen Coons, Ph.D.

¹² Furlong WJ, Feeny DH, Torrance GW, Barr RD. The Health Utilities Index (HUI) system for assessing health-related quality of life in clinical studies. Ann Med. 2001;33(5):375-84.

¹³ Feeny DH, Torrance GW, Furlong WJ. Health Utilities Index. In: Spilker B, ed. Quality of Life and Pharmacoeconomics in Clinical Trials. 2nd ed. Philadelphia: Lippincott-Raven; 1996.

¹⁴ Kaplan RM, Anderson JP. A general health policy model: update and applications. Health Serv Res. 1988;23(2):203-35.

¹⁵ Brazier J, Usherwood T, Harper R, Thomas K. Deriving a preference-based single index from the UK SF-36 Health Survey. J Clin Epidemiol. 1998;51(11):1115-28.

Task Forces Index