OBJECTIVES

In comparative effectiveness research Simulated Treatment Comparisons (STCs) are becoming increasingly common in the absence of head-to-head trials. STCs use estimates from limited IPD to adjust for covariate imbalance between trials, however the uncertainty from these estimates is generally ignored when estimating relative treatment effects. This study demonstrates the need to account for this uncertainty when conducting STCs. We introduce an STC method that accounts for the uncertainty due to covariate adjustment, and demonstrate its effectiveness via simulation.

METHODS

We simulated two single arm studies (N=300 for both), each containing age and overall survival. We assume study 1 has individual patient data available, and study 2 only has aggregate age data and a digitized Kaplan-Meier curve. We compute a covariate adjustment term based on the mean age difference between the studies and the age coefficients from fitting a parametric survival model to the observed study 1 IPD. We then estimate the variance of this adjustment term via bootstrapping and incorporate this uncertainty into a Bayesian STC model which estimates the relative treatment effect for the two study datasets converted to a digitized Kaplan-Meier format.

RESULTS

The proportion of 95% CrI’s that captured the true treatment effect was 86.8% without error propagation, whereas 92.0% of CrI’s captured the true treatment with error propagation. 94.9% of CrI’s contained the true treatment effect when using survival regression with the complete IPD.

CONCLUSIONS :

Failing to account for uncertainty from covariate adjustment when conducting simulated treatment comparisons generally leads to underestimating the uncertainty of relative treatment effects. This method better captures the uncertainty introduced when conducting an STC.