Presentation (CTI)

OBJECTIVES: External control arm (ECA) studies rely on covariate adjustment, yet real-world evidence (RWE) studies often follow prognostic importance or machine learning (ML) approaches for variable selection rather than causal relevance. These practices may introduce structural bias, particularly when colliders are included. This study examined how different selection strategies impact treatment-effect estimation when comparing a single-arm trial (SAT) with an ECA.
METHODS: A total of 2,000 datasets, including an SAT (n=80) and an ECA (n=800), were simulated. Each dataset included two true confounders (c1,c2), two prognostic variables (p1,p2), an instrumental variable (z1), a collider (coll1), an unmeasured confounder (u1), and two noise variables. Treatment depended on confounders, unmeasured confounder, and instrumental variable. Outcome followed a logistic model including treatment, confounders, prognostic variables, and the unmeasured confounder. The true treatment effect was set at log(OR) = -0.431. Four selection strategies were evaluated: 1) a directed acyclic graph (DAG)-based set including only true confounders; 2) prognostic strength ranking based on outcome associations; 3) cross-validated logistic LASSO; and 4) random forest (RF) variable importance. For each dataset, treatment effects were estimated using naïve logistic regression, adjusted regression, and stabilized ATT inverse-probability weighting. Bias, RMSE, and 95% interval coverage were summarized.
RESULTS: DAG-based adjustment resulted in the lowest bias and RMSE, with coverage close to nominal levels (0.94). Prognostic ranking, LASSO, or RF approaches frequently included colliders or instrumental variables, leading to bias amplification (e.g., prognostic approach bias -0.243; LASSO bias -0.275) and reduced coverage. Weighted estimators improved performance (lower bias, greater coverage) but were less accurate (more deviation from true effect) than the DAG-based sets.
CONCLUSIONS: Adjustment sets grounded in causal theory, where back-door paths are blocked with minimal sufficient adjustment sets, yielded more accurate and stable estimates. These findings support the use of causal frameworks when designing ECAs and conducting comparative analyses in RWE research.