The ISPOR Health Outcomes Metrics Special Interest Group
The field of pharmacoeconomics and outcomes research benefits greatly from its interdisciplinary nature; we adapt and adopt methods from a variety of disciplines including biostatistics, econometrics, psychometrics, operations research, informatics and epidemiology. Researchers in our field confront many data challenges, including missingness, non-normality, joint determination of multivariate outcomes (cost and effectiveness), clustering of observations and endogenous treatment selection. Other fields that contribute to our analytic toolset have developed approaches to deal with these issues, including: multiple imputation, generalized linear models (GLM), two-part models, structural equations, hierarchical models, propensity scores and instrumental variables, to name a few. Given that the field of pharmacoeconomics and outcomes research guides policy/decision makers, in addition to the usual concerns about internal validity and causal inference, we need to pay close attention to generalizability and quantification of uncertainty. To address these issues, we have adopted methods from the decision sciences including predictive simulation and Bayesian inference. All of these “solutions” come with their own sets of assumptions and limitations.
It’s a challenge to keep up with methodological advances in the fields in which we are trained, not to mention becoming familiar enough with the methods of other disciplines that need to be used to address different research questions in our field. The ISPOR Health Outcomes Metrics Special Interest Group (HOM) was created to “identify, highlight, and promote appropriate and novel approaches to health outcomes metrics used in pharmacoeconomics and outcomes research.” (http://www.ispor.org/sigs/HOM.asp). The HOM SIG currently has two active working groups, the Health Econometrics Working Group (HEWG) and Health Outcomes Metrics Education Working Group (HOME-WG) to help achieve its goal.
The ISPOR Health Econometrics Working Group (HEWG) was formed with the interrelated goals of promoting transparency of quantitative methods and encouraging comparative modeling (i.e., testing the sensitivity of results to modeling methods). While canned software programs are available for many of the standard econometric/statistical methods, the newer methods often lack the codes or they may be widely dispersed or the existing codes may not be user-friendly. This in turn may discourage researchers in our field who are not programming-savvy to apply those new methods in their research. Furthermore, even the canned software can be customized in variety of ways to address specific research questions, which may be inscrutable to the inexperienced user, and therefore appear daunting. The HEWG’s goal of promoting transparency in methods is meant to facilitate replication of those methods in different research contexts. Transparency of methods is mutually beneficial to both authors and consumers of research. Transparency lends credibility and allows for the rapid dissemination and adaptation (with full attribution of course!) of innovative approaches. Some journals (for example, the Journal of Applied Econometrics) have gone so far as requiring that authors deposit both the data and code necessary for the research community to replicate results (see: http://onlinelibrary.wiley.com/journal/10.1002/ (ISSN)1099-1255/homepage/ForAuthors.html).
Transparency is also instrumental in achieving the second HEWG goal of promoting comparative modeling. Decision makers who rely on our research should be made aware of the sensitivity of our conclusions to the modeling choices made. Comparative modeling is relatively simple when off-the-shelf methods are used, but when we develop “custom-made” solutions to our data challenges, the need to make code available to the research community is particularly strong.
Our main vehicle to achieve these first two goals has been the creation and maintenance of the ISPOR Health Outcomes Metrics Index of Open Source Code (http://www.ispor.org/OpenSourceIndex/Index.aspx). The HOM Index includes brief descriptions and hyperlinks to open-source code that can be downloaded for free. Currently, the HOM Index is arranged topically: treatment effects, health care costs, health care utilization, quality of life and utility, censoring and survival and “other.” We encourage ISPOR members to submit their own code or suggest links to others’ code they have found helpful in their own research by sending an email to: firstname.lastname@example.org. In the near future, HEWG will develop a strategy with the editorial staff of Value in Health to encourage (but not require) authors to submit open source code to the HOM Index.
The ISPOR Health Outcomes Metrics Education Working Group (HOME-WG) was created to “recognize, promote and encourage the development of outstanding online didactic material and educational resources in health outcomes metrics for use in pharmacoeconomics and outcomes research” (http://www.ispor. org/sigs/HOM_Education-Working-Group.asp). Notwithstanding the strong presence of ISPOR’s student members, it’s probably safe to say that it’s been at least a little while since the majority of us have been in graduate school. Nevertheless, in our rapidly evolving profession, each of us plays the roles of teacher and learner every day. Whether we are academics in the classroom, managers explaining results to opinion leaders, or scientists describing our methods to clients and colleagues, there is some element of didactic teaching involved in what we do on a daily basis.
Many excellent webinars, course syllabi, slide presentations, book chapters and homework exercises are widely available on the Internet – unfortunately, they are also widely dispersed. The primary aim of HOME-WG is to develop and maintain a centralized, searchable index of excellent educational materials focusing on the methods of pharmacoeconomics and outcomes research on the ISPOR website. The HOME-WG leadership group is currently developing the initial version of the index. We will then invite the ISPOR membership to contribute their own material or provide links to material they have found useful in the past. Ultimately, the HOME-WG will take steps to publicly recognize particularly excellent resources. The Health Outcomes Metrics Special Interest Group can only achieve its goals with the active support of ISPOR members. We strongly encourage you to join HEWG and/or HOME-WG (see: http://www. ispor.org/sigs/sigsindex.asp for steps on how to join) and to contribute your own code and educational materials. Please stay tuned as these exciting and useful endeavors progress. [Health Outcomes Metrics SIG Leadership Group: Anirban Basu, PhD, Bijan J. Borah, PhD, Benjamin M. Craig, PhD, Jalpa Doshi, PhD, David J. Vanness, PhD]