OBJECTIVES

There is increased interest in estimating heterogeneous responses to medical interventions in the context of randomized trials and observational studies. New approaches, drawing heavily from concepts in machine learning, have been developed that allow researchers to examine heterogeneity in a data-driven, but statistically principled, way. This research highlights some of these developments and provides an example of their application within the context of pharmacoeconomics and outcomes research.

METHODS

Four novel approaches to heterogenous treatment effect estimation are described. These approaches can be categorized as based on decision trees or based on regularizing interaction terms. Two decision tree approaches, Bayesian Additive Regression Trees (BART) and Causal Forests (CF), and two regularized interactions approaches, Bayesian LASSOplus (BLASS) and elastic-net GLMs (EL-GLM), are compared in the context of an observational study examining the effect of recommended diagnostic testing and costs for Americans with diabetes.

RESULTS

Three of the approaches show broad agreement about the magnitude of the average treatment effect (BART $1,697; CF $1,942; EL-GLM $1,324) although there is significant uncertainty in all of these estimates. The BLASS approach suggests no effect of recommended diagnostic testing on expenditures. The approaches also show disagreement about which variables predict effect heterogeneity. For example, while all methods suggest co-morbidities are important determinants of heterogeneity, the CF approach suggests sex is also important whereas BART puts more emphasis on an individual’s race/ethnicity.

CONCLUSIONS

New methods for heterogenous treatment effect estimation are being developed that have the potential to generate novel insights for HEOR researchers. The results of this analysis suggest choice of estimation approach can yield different inferences, therefore it is advised researchers consider using multiple techniques before drawing conclusions. Future studies could examine the use of these methods in situations beyond the cross-sectional, observational setup described here.