OBJECTIVES: Using a historical control or external control arm (ECA) to augment or replace a concurrent control arm in a randomized trial is a hot topic given the challenge of patient recruitment in rare diseases or during COVID-19 pandemic. FDA released several draft guidance in 2021 on effectiveness and safety submissions using real-world evidence. While the guidance focuses mainly on elements of study design and data source selection, there is a lack of consensus in the selection of appropriate statistical methods when constructing an ECA. This study aims to discuss rigorous statistical methodology for ECA-supported trial in regulatory or HTA submissions.

METHODS: Targeted literature reviews of statistical simulations comparing methods for ECA in statistical journals were performed. The articles compare commonly used ECA-construction and analysis methods were selected and summarized, including but not limited to propensity-score (PS) based- matching, weighting, stratification, and PS plus Bayesian integrated approaches.

RESULTS: Type I error, power, bias, and coverage probability are common criteria used to compare different methods. When imbalances only exist in known baseline covariates and the outcome distribution are the same between the trial concurrent control and ECA, PS method alone or paired with commensurate prior yield almost unbiased estimates, good Type I errors, and coverage probability. PS plus Bayesian approaches have wider interval width and lower power compared with PS only methods. When there is a change in the outcome distribution over time, PS (matching or IPTW) and commensurate prior integrated method yield smallest biases among all methods.

CONCLUSIONS: PS and Bayesian integrated methods outperformed the PS only methods in terms of bias and type I error when outcome distribution changed with current trial control. A “sweet spot” that balances all criteria through trial-specific simulations could provide the ideal setting of trial analyses plan based on specific trial design and scenarios.