Single-Arm Studies: Are All Created Equal?

OBJECTIVES: Randomized controlled trials (RCTs) are the gold standard for evaluating effectiveness of interventions. Interventional single-arm trials (SATs) are increasingly being considered, despite a lack of agreement on their validity and position in the hierarchy of evidence. Notably, it is unclear whether SATs are superior to observational single-arm studies (case series). We investigated whether there are systematic differences in outcome and between-study heterogeneity for SATs compared with case series.

METHODS: We conducted a pragmatic literature review for systematic reviews (SRs) of pharmacological interventions including single-arm studies. A single reviewer identified SRs and extracted primary study characteristics and outcome data. For each SR, meta-analysis of dichotomous outcomes was conducted, with sub-group analysis of the included SATs versus case series. To investigate whether statistical heterogeneity was explained by clinical heterogeneity or indicated bias, clinically different primary studies were removed in a sensitivity analysis.

RESULTS: 13 SRs were included. When primary studies were sub-grouped by study design, there was no significant difference for SATs versus case series across SRs (risk difference -0.02, 95% CI 0.09, 0.05). There were high levels of between-study heterogeneity within both SATs (median I²: 55%) and case series (median I²: 77%). When clinically heterogenous studies were removed, effect size tended to be greater for case series, but not significantly so (risk difference -0.071, 95% CI -0.161, 0.019). Levels of within-group statistical heterogeneity remained high, suggesting that bias may have been a moderator of effect in both SATs and case series.

CONCLUSIONS: There do not appear to be systematic differences in outcome between SATs and case series. However, levels of heterogeneity in effect size are high within both designs, even after attempts to reduce clinical heterogeneity, indicating that bias may have an impact on outcomes. Future work should utilize larger samples and additional methods to further clarify the relative validity of single-arm designs.