WHEN DOUBLY ROBUST IS NOT ROBUST ENOUGH- NONPARAMETRIC MATCHING METHODS UNDER TREATMENT HETEROGENEITY

Author(s)

Shao H1, Stoecker C2, Shi L1
1Tulane University School of Public Health and Tropical Medicine, New Orleans, LA, USA, 2Tulane University School of Public Health and Tropical Medicine, NEW ORLEANS, LA, USA

OBJECTIVES: Recent technological advances have increased the ability to deliver precision-targeted individualized treatment plans. This approach concentrates treatments among patients most likely to benefit from a particular therapy. This presents methodological challenges for evaluators as the average treatment on the treated (ATT) is likely to be substantially larger than the average treatment effect (ATE) considered across the entire population. This study aimed to explore the most accurate non-parametric matching methods for estimating the ATT among a population with treatment heterogeneity.  METHODS: Both Monte Carlo simulation and semi-simulation using actual data from the Medical Expenditure Panel Survey (MEPS) 2012 were conducted. We simulated three propensity score distributions each with three treatment heterogeneity correlations. Within each of these nine scenarios, we created 1000 datasets each with 500 patients. Mean squared predicted error for OLS, one-to-one matching, k-nearest-neighbor matching, propensity score weighting, kernel matching, and local linear matching were calculated and compared to known truth. For each matching estimator, the ATT was estimated with both direct mean comparison and weighted OLS using the propensity scores in a weighting function. RESULTS: All matching methods showed improvement (14%~69%) compared to using an estimator based on OLS alone. Kernel matching provided the best theoretical improvement over OLS (69%). Because the efficiency of kernel matching relies on the choice of bandwidth, we applied leaving-one-out algorithm to identify the bandwidth. Kernel matching with optimized bandwidth yielded a 61% improvement over OLS, and outperformed the other matching methods. In addition, our simulated results suggested that applying OLS after matching yielded worse results than a direct mean comparison after matching. CONCLUSIONS: Kernel matching was identified as the most accurate methods in estimating ATT, and the leaving-one-out algorithm was a reliable method in selecting bandwidth. Matching provides improvement over OLS, but caution should be exercised when combining matching with regression techniques.

Conference/Value in Health Info

2017-05, ISPOR 2017, Boston, MA, USA

Value in Health, Vol. 20, No. 5 (May 2017)

Code

PRM93

Topic

Methodological & Statistical Research

Topic Subcategory

Confounding, Selection Bias Correction, Causal Inference, Modeling and simulation

Disease

Multiple Diseases

Explore Related HEOR by Topic


Your browser is out-of-date

ISPOR recommends that you update your browser for more security, speed and the best experience on ispor.org. Update my browser now

×