Comparing Causal Inference Methods With and Without Probabilistic Bias Analysis in the Presence of Selection Bias in External Control-Arm Studies
Author(s)
Tyler Richter, PhD, Kael Wherry, PhD.
Medtronic, Inc., Mounds View, MN, USA.
Medtronic, Inc., Mounds View, MN, USA.
OBJECTIVES: Comparative effectiveness of treatment studies using external control arm (ECAs) designs face challenges in identifying suitable comparator cohorts and selecting appropriate analytical methods. In particular, bias due to unmeasured differences in how patients are selected into treated and external control groups may threaten causal validity. This study uses simulations and real-world data (RWD) to compare the performance of multiple causal inference methods, with and without probabilistic bias analysis (PBA), in the presence of selection bias in an observational ECA design.
METHODS: We will conduct both a simulation and applied analysis within an ECA framework. The data-generating process will include a binary treatment, continuous outcome, and multiple observed and unobserved confounders. Selection bias will be introduced using a logistic model in which inclusion into the analytic sample depends on the outcome and an unobserved variable (representing characteristics such as disease severity). In the applied analysis, the outcome is the change in systolic blood pressure. The treated group will consist patients from the Global Symplicity Registry who received renal denervation. The controls will be drawn from the Australian general practice database, PATRON, using inclusion criteria based on blood pressure, clinical history, and demographics.
We will compare inverse probability of treatment weighting, propensity score (PS) matching, and doubly robust estimation, with and without PBA, in the presence of selection bias. A linear mixed model will be fit for the outcome and PS will be estimated using logistic regression with baseline demographics, clinical, and social-behavioral covariates. We will estimate the average treatment effect (ATE) and average treatment effect on the treated (ATT). ATE estimation will use full optimal PS matching, whereas ATT will use 1:2 nearest neighbor (without replacement) matching with calipers of 0.2*SD(logit(PS)). Bias parameters in the PBA and the magnitude of selection bias will be informed by the literature.
RESULTS:
CONCLUSIONS:
METHODS: We will conduct both a simulation and applied analysis within an ECA framework. The data-generating process will include a binary treatment, continuous outcome, and multiple observed and unobserved confounders. Selection bias will be introduced using a logistic model in which inclusion into the analytic sample depends on the outcome and an unobserved variable (representing characteristics such as disease severity). In the applied analysis, the outcome is the change in systolic blood pressure. The treated group will consist patients from the Global Symplicity Registry who received renal denervation. The controls will be drawn from the Australian general practice database, PATRON, using inclusion criteria based on blood pressure, clinical history, and demographics.
We will compare inverse probability of treatment weighting, propensity score (PS) matching, and doubly robust estimation, with and without PBA, in the presence of selection bias. A linear mixed model will be fit for the outcome and PS will be estimated using logistic regression with baseline demographics, clinical, and social-behavioral covariates. We will estimate the average treatment effect (ATE) and average treatment effect on the treated (ATT). ATE estimation will use full optimal PS matching, whereas ATT will use 1:2 nearest neighbor (without replacement) matching with calipers of 0.2*SD(logit(PS)). Bias parameters in the PBA and the magnitude of selection bias will be informed by the literature.
RESULTS:
CONCLUSIONS:
Conference/Value in Health Info
2025-11, ISPOR Europe 2025, Glasgow, Scotland
Value in Health, Volume 28, Issue S2
Code
MSR57
Topic
Clinical Outcomes, Methodological & Statistical Research, Study Approaches
Topic Subcategory
Confounding, Selection Bias Correction, Causal Inference
Disease
Cardiovascular Disorders (including MI, Stroke, Circulatory), No Additional Disease & Conditions/Specialized Treatment Areas