BACKGROUND: In health technology assessment (HTA), data from randomised controlled trials (RCTs) are synthesised to evaluate effectiveness of health technologies. More recently, HTA agencies have acknowledged that including observational studies in the synthesis, for example single-arm studies, may be appropriate where there is little RCT evidence.

OBJECTIVE: We aim to compare methods which synthesise aggregate data from RCTs and single-arm studies, via application to an illustrative example in rheumatoid arthritis (RA) and a simulation study.

METHODS: We consider contrast-based methods proposed by Begg and Pilote and arm-based methods by Zhang et al. We apply methods to data from a published HTA, including three RCTs and 11 single-arm studies, evaluating biologic agents as second-line treatments for RA, and reporting the number of participants achieving the American College of Rheumatology (ACR20) response criteria. We also perform a simulation study with scenarios varying (i) the proportion of RCTs and single-arm studies in the synthesis (ii) the magnitude of bias, and (iii) between-study heterogeneity.

RESULTS: Applying random-effects meta-analysis solely to RCT data produced a highly uncertain estimate for the effect of biologic treatment versus placebo on achieving ACR20; OR= 3.40 (0.10, 105.10). The original method by Begg and Pilote, combining data from RCTs and single-arm studies, produced improvement in precision; OR= 3.06 (2.42, 3.90). The hierarchical commensurate and power prior methods by Zhang et al produced more modest improvements in precision; OR= 2.54 (0.61, 8.70) and OR= 2.58 (0.68, 9.32), respectively. Our simulation study showed that the arm-based methods performed better when there were few RCTs, and between-study heterogeneity differed between the two data sources.

CONCLUSIONS: The hierarchical power and commensurate prior methods provide the most robust approach to synthesising aggregate data from RCTs and single-arm studies, balancing the need to account for bias and differences in between-study heterogeneity, whilst reducing uncertainty in estimates.