ASSESSING THE GENERALIZABILITY OF CLINICAL TRIALS WITHOUT INDIVIDUAL-LEVEL TRIAL DATA
Author(s)
Wang WJ1, Bansal A2, Bennette C3, Basu A2
1The Comparative Health Outcomes, Policy, and Economics (CHOICE) Institute, University of Washington, Seattle, WA, USA, 2University of Washington, Seattle, WA, USA, 3Flatiron, Seattle, WA, USA
OBJECTIVES : When individual-level data from both a clinical trial and the real-world data (RWD) are available, propensity score (PS) approaches can directly assess the generalizability of trials based on common observed baseline factors. When only summary data from trials are available, clinical trial samples can be simulatedby using the observed correlations among baseline factors in the RWD. Such simulated trial data may lead to spurious generalizability estimates due to true differences in correlation structures between trial and RWD. METHODS We developed an algorithm that uses summary baseline data of a trial and iteratively uses copula and resampling methods (using weights from standard PS approaches) to approximate the true correlation matrix in the trial. Stopping rule for the algorithm was based on the threshold-based convergence of PS model coefficients across iterations. Validation was performed using Monte-Carlo simulation with several scenarios. The final correlation matrix after convergence was used to simulate the final individual–level trial baseline data. Standard PS approach was applied to generate PS distributions for the trial and RWD to assess the generalizability using standardized mean difference, B-index, and Kolmogorov-Smirnov statistic. RESULTS Across all of our simulated scenarios, we found that the algorithm was successful in closely reproducing the joint distribution of baseline characteristics of the trial data using only summary data from the trial and the RWD. On average, the algorithm converged within 300 iterations. We found that external validity could be assessed using any of the generalizability metrics produced using the trial data from our algorithm. CONCLUSIONS : Our algorithm is a feasible way to simulate individual-level clinical trial data when only summary data from the trials are available, which could be used to assess the generalizability of clinical trials given participants’ characteristics and inform decision making around the applicability of trial results to a real-world population.
Conference/Value in Health Info
2019-05, ISPOR 2019, New Orleans, LA, USA
Value in Health, Volume 22, Issue S1 (2019 May)
Code
PNS218
Topic
Clinical Outcomes, Methodological & Statistical Research, Real World Data & Information Systems
Topic Subcategory
Artificial Intelligence, Machine Learning, Predictive Analytics, Comparative Effectiveness or Efficacy, Reproducibility & Replicability
Disease
No Specific Disease