OBJECTIVES

Real-world evidence (RWE) is increasingly considered alongside trial data to estimate relative effectiveness of treatments. However, using RWE reliably can be challenging. We examined the feasibility of using RWE to assess relative clinical effects of left atrial appendage closure (LAAC) with the WATCHMAN device and oral anticoagulants (OACs) for stroke risk reduction in nonvalvular atrial fibrillation to inform decisions on conducting a cost effectiveness analysis.

METHODS

A systematic literature review identified potential RWE sources for OACs, which were compared to EWOLUTION, a prospective registry on LAAC with the WATCHMAN device, to assess the feasibility of conducting a statistical indirect comparison. Homogeneity of the evidence base was evaluated on study design, patient characteristics, endpoint derivation and reported results. Published risk equations were used to re-estimate event rates within studies to determine the feasibility of adjusting for differences in patient characteristics across studies using reported aggregate data.

RESULTS

We identified three meta-analyses, three prospective registries and five retrospective data analyses. Patients enrolled in EWOLUTION were older (25.6% ≥80) and had higher stroke risk (73.1% with CHA₂DS₂-VASc ≥4) and more comorbidities than OAC patients. Inclusion criteria and endpoint definitions across studies were importantly different. Calculated risks using published equations did not sufficiently reproduce observed event rates. Compared to observed rates, predicted annual stroke rates differed by >50%, death rates were underestimated by >15% and major bleeding rates were underestimated by >20%.

CONCLUSIONS

Risk adjustment analyses using aggregate data yielded low levels of reproducibility, which led to widely different estimates of clinical effect. Coupled with differences in study designs, it is inadvisable to compare the real-world safety and effectiveness of WATCHMAN to OACs using existing real-world studies, even when adjusting for differences in patient characteristics. Attempts should be made to improve consistency of reporting to fully realize the value of RWE in clinical and health economic evaluations.