OBJECTIVES : Many methods have been proposed for incorporating historical data to act as a synthetic control for clinical trials. In this work, comparisons are made between hierarchical commensurate power prior model (CPP) versus a “naïve” mixed-effect model (MEM).

METHODS : Synthetic control groups are generated by applying the models to individual patient-level data (IPD). The focus has been on time-to-event endpoints. The CPP model is developed with Weibull assumption and compared with an existing MEM with the same assumption. To be able to evaluate the methods based on true parameter values, we use simulated data based on multiple scenarios. These scenarios are constructed based on three factors: the sample size, the ratio of concurrent control to treatment arm sizes, and the compositions of the historical control data (i.e. consistent or inconsistent with the concurrent control or a mixture of both types). Effect of each of these three factors on the performance of CPP and MEM models are analyzed in terms of estimate bias and credible intervals.

RESULTS : When closely matched historical data is available, CPP resulted in more accurate estimates and smaller credible intervals than MEM for all data sizes and concurrent control ratios. However, in the case where historical data is either a mixture of consistent and inconsistent or only inconsistent evidence, CPP estimates are biased towards the inconsistent parameters, especially for smaller sample sizes. Without any concurrent control, MEM yields very large credible intervals and random estimates. On the other hand, CPP produces small credible intervals which are accurate when the historical control data is consistent.

CONCLUSIONS : CPP is more accurate in presence of consistent historical data even if no concurrent control data is available. However, when evidence for commensurability is weak, a “naïve” MEM would produce more precise estimates of the model parameters.