OBJECTIVES: To provide further evidence of the value of clinical expertise alongside data-driven propensity score (PS) model selection in observational studies.

METHODS: Analysis utilized data from a multiyear, prospective, observational study that evaluated the safety of a new surgical procedure to standard procedure in 987 patients (N1=280, N2=707). Clinical experts identified baseline confounders for the outcomes and classified into tiers: Tier 1=definite risk factors, Tier 2=probable confounders, and Tier 3=potential instruments. Five logistic PS matching models (1:1 ratio) were developed from combinations of confounders. Tier 1 confounders were forced in all models and Tier 2 and 3 confounders were either forced or included via stepwise selection (entry/exit P-value: ≤0.25/> 0.25). Final model selection was based on maximizing proportions matched between cohorts and balance across all confounders (ie, mean and maximum standardized differences [meanSTDIFF and maxSTDIFF] <0.20).

RESULTS: The clinical experts identified 14 potential baseline confounders and classified 6 as Tier 1, 3 as Tier 2, and 5 as Tier 3. The proportion matched between cohorts ranged from 92.1% (Model 2) to 95.0% (Model 4) across the 5 models. Models 3, 4, and 5 achieved balance across all 14 confounders, as meanSTDIFF ranged from 0.0383 (Model 4) to 0.0530 (Model 5), and maxSTDIFF from 0.1023 (Model 3) to 0.1616 (Model 4). Among Tier 1 confounders, the maxSTDIFF ranged from 0.0608 (Model 4) to 0.0686 (Model 5). After clinical and analytical considerations, Model 4 was selected as the final model based on the highest match proportion, balance across all confounders (meanSTDIFF and maxSTDIFF<0.20), lowest meanSTDIFF, and lowest maxSTDIFF for Tier 1 confounders.

CONCLUSIONS: This analysis demonstrates the added value of clinical expertise in PS matching model development and selection. Clinical expertise is important to instilling confidence to a largely data-driven approach for the interpretation of the results in the medical community.