Consistency Between Comparative Efficacy Estimates from Target Trial Emulations and Their Origin Target Trials: A Meta-Epidemiological Study

OBJECTIVES: To compare comparative effect estimates of target trial emulations (publications) based on real-world data (RWD) and their origin target trial counterparts. To examine select factors that determine consistency or inconsistency between the two.

METHODS: We performed a systematic literature review searching for TTEs based on RWD that emulated an existing and completed randomized clinical trial (RCT), and extracted comparative effects from all TTE and their origin target trial.

RESULTS: Of 567 abstracts screened and 236 TTE publications identified, a total of 33 publications reporting on 43 TTE were identified. The majority of publications (n=29, 87%) contained a single trial being emulated. Fourteen of the publications performed TTE for one outcome, and the remainder ranged from 2 to 8 outcomes. The most commonly assessed therapeutic area was oncology (N = 11, 33%), followed by cardiology (n=7, 21%) and rheumatology (n=3, 9%). The most common geography source of data for emulation was from the US (n=12, 36%), followed by the UK (n=6, 18%) and Sweden (n=4, 12%). For their primary outcome, most trials reported hazard ratios (HR) (N = 14, 42%), followed by risk ratios (RR) (N = 7, 21%) and (standardized) mean differences (SMD/MD) (N = 4, 12%). Of the 18 trials (39 TTEs) reporting relative effects (HR or RR), only one had inconsistent treatment effects and non-overlapping 95% confidence intervals between the TTE and origin target trial. A majority (3 out of 4 publications, 6 out of 9 TTEs) that analyzed SMD/MD yielded moderate to strong inconsistencies. The sample size was too small to infer whether disease area or country of emulation were confounding factors.

CONCLUSIONS: Consistency between TTE and origin target trial results appear difficult to obtain for continuous effect measures, but feasible for relative effect measures.