OBJECTIVES

When estimating treatment effects using real-world data, we risk introducing bias when declaring (specifying) a particular data generation process (DGP) model as if it were known. For this study, we will evaluate treatment effect estimates published from randomized control trials and compare these to estimates obtained by applying data-adaptive methods, particularly those offered by targeted learning (TL), on a cohort of COVID-hospitalized patients.

METHODS

We evaluated published clinical trials of Remdesivir-treated hospitalized patients and estimates of hospital length of stay (LoS) and symptom severity reduction. We made an effort to match as close as possible the inclusion/exclusion criteria, clinical outcome definitions, and other trial conditions and elements to evaluate treatment effects in a real-world cohort of 6,593 patients hospitalized with COVID (62% received Remdesivir within seven days of admission). Analytically, we applied targeted learning to estimate the average treatment effects while adjusting for potential confounders in the real-world cohort.

RESULTS

Clinical trials indicate both reductions in LoS and COVID severity among the treated patients. Targeted learning estimates mirror clinical trial results in a decrease in severity (on an ordinal 3-point scale) (p=0.0028) but not for LoS (increased, p<0.001) among the treated. Both TL models implement ensemble SuperLearner estimation of g (treatment mechanism) to adjust for potential confounding.

CONCLUSION

Targeted learning provides a useful framework for estimating effects in real-world data. However, much care still needs to go into cohort definition and outcome definition for targeted learning to support scientific discovery. Future work may also incorporate longitudinal data and decision opportunity sequences (dynamic and stochastic treatment strategies). We also plan to compare the TL methods with the conventional approaches (e.g., propensity scores) in various clinical settings to understand when TL offers the advantage over traditional methods.