WHEN DOUBLY ROBUST IS NOT ROBUST ENOUGH- NONPARAMETRIC MATCHING METHODS UNDER TREATMENT HETEROGENEITY
Author(s)
Shao H1, Stoecker C2, Shi L1
1Tulane University School of Public Health and Tropical Medicine, New Orleans, LA, USA, 2Tulane University School of Public Health and Tropical Medicine, NEW ORLEANS, LA, USA
OBJECTIVES: Recent technological advances have increased the ability to deliver precision-targeted individualized treatment plans. This approach concentrates treatments among patients most likely to benefit from a particular therapy. This presents methodological challenges for evaluators as the average treatment on the treated (ATT) is likely to be substantially larger than the average treatment effect (ATE) considered across the entire population. This study aimed to explore the most accurate non-parametric matching methods for estimating the ATT among a population with treatment heterogeneity. METHODS: Both Monte Carlo simulation and semi-simulation using actual data from the Medical Expenditure Panel Survey (MEPS) 2012 were conducted. We simulated three propensity score distributions each with three treatment heterogeneity correlations. Within each of these nine scenarios, we created 1000 datasets each with 500 patients. Mean squared predicted error for OLS, one-to-one matching, k-nearest-neighbor matching, propensity score weighting, kernel matching, and local linear matching were calculated and compared to known truth. For each matching estimator, the ATT was estimated with both direct mean comparison and weighted OLS using the propensity scores in a weighting function. RESULTS: All matching methods showed improvement (14%~69%) compared to using an estimator based on OLS alone. Kernel matching provided the best theoretical improvement over OLS (69%). Because the efficiency of kernel matching relies on the choice of bandwidth, we applied leaving-one-out algorithm to identify the bandwidth. Kernel matching with optimized bandwidth yielded a 61% improvement over OLS, and outperformed the other matching methods. In addition, our simulated results suggested that applying OLS after matching yielded worse results than a direct mean comparison after matching. CONCLUSIONS: Kernel matching was identified as the most accurate methods in estimating ATT, and the leaving-one-out algorithm was a reliable method in selecting bandwidth. Matching provides improvement over OLS, but caution should be exercised when combining matching with regression techniques.
Conference/Value in Health Info
2017-05, ISPOR 2017, Boston, MA, USA
Value in Health, Vol. 20, No. 5 (May 2017)
Code
PRM93
Topic
Methodological & Statistical Research
Topic Subcategory
Confounding, Selection Bias Correction, Causal Inference, Modeling and simulation
Disease
Multiple Diseases