OBJECTIVE: We compare the performance of propensity score weighting (PSW) with the optimization-based stable balancing weights (SBW) technique for creating an external control arm to a clinical trial using real-world data in terms of (a) the post-weighting effective sample size achieved (larger being better) and (b) the distribution of standardized mean differences (SMDs) for covariates between the treated and control groups (smaller being better).

METHODS: We analyzed prospectively-collected data on 167 patients undergoing left-sided colorectal reconstructions with the ECHELON CIRCULAR™ Powered Stapler (PCS group) and retrospectively-collected data on 5,333 patients undergoing left-sided colorectal reconstructions with manual circular staplers collected from the Premier Healthcare Database (PHD group). We sought to weight to the PHD group such that it mimicked the PCS group on 10 weighting covariates: surgical approach (open, laparoscopic, or robotic), indication for surgery (colorectal carcinoma, colorectal polyps or polyp syndrome, diverticulitis, inflammatory bowel disease, or other), age, sex, Hispanic ethnicity, teaching vs. non-teaching hospital, comorbid diabetes, comorbid hypertension, and commercial vs. government insurance. We computed PSW using multivariable logistic regression. We implemented SBW performing a grid search to find the lowest SMD setting at which no effective sample size was lost (0.002).

RESULTS: Before weighting, there were substantial imbalances between the PCS and PHD groups on the 10 weighting covariates, with an average (range) SMD value of 0.35 (0.01—2.35). After applying PSW, the effective sample size was 1,633 and the average (range) SMD value of 0.01 (0.00—0.03). In contrast, SBW resulted in an effective sample size of 1,882 and average (range) SMD values of 0.00 (0.00—0.00).

CONCLUSIONS: In this analysis, SBW outperformed PSW on both post-weighting effective sample size and post-weighting covariate balance between the treated and control groups. Optimization-based techniques for observational comparative effectiveness research show substantial promise and warrant further adoption and continued evaluation.