OBJECTIVES: In orphan and rare diseases, single-arm trials are common given the impracticability, if not impossibility, of randomized controlled trials. In these settings, an external control (EC) can be employed to compare against the single-arm trial to estimate treatment effects, though minimizing potential biases to interpret these effects is critical. We sought to compare different methods for causal inference in simulated data sets with measured (included from the model) and unmeasured (excluded) confounders.

METHODS: In the simulated data, three types of outcomes (continuous, binary, and time-to-event) were compared between trial and EC arms. Two measured baseline covariates (confounders) were unbalanced between the two arms. For each outcome, the two scenarios for relationship between unmeasured confounder and observed variables were: A) directly associated with one baseline covariate and the outcome; and B) directly associated with one baseline covariate, the outcome, and treatment assignment. In 3,000 random samples (100 or 200 patients per sample), we used g-computation, propensity score-based weighting, and targeted maximum likelihood estimation (TMLE) to estimate treatment effects based on observed outcomes and measured covariates.

RESULTS: In scenario A, the average estimates from all proposed methods were similar to the true effect (e.g. 1.14 to 1.19 vs 1.15 for the binary outcome), but g-computation had the smallest mean squared error (MSE), as well as a reasonable coverage (0.91-0.94) as measured by 95% confidence interval in different settings. In scenario B, most of results were similar with those in scenario A, except in one setting of continuous outcome where the estimates were more discrepant from the true effect.

CONCLUSIONS: Treatment effects estimated by g-computation, propensity score-based weighting, and TMLE appear to be reasonable for EC trials, even if confounders are not completely measured. Of the proposed approaches, evidence suggests that g-computation is a preferable approach producing relatively unbiased estimates.