Validation of AI-Generated Synthetic Data for Outcomes Research: A Clinico-Genomics Case Study
Author(s)
Theodorou B1, Das A2, Zhang NJ3, Sun J1, Yay Donderici E4
1Medisyn Inc., Las Vegas, NV, USA, 2Guardant Health, Cambridge, MA, USA, 3Guardant Health, Palo Alto, CA, USA, 4Guardant Health, Burlingame, CA, USA
OBJECTIVES: The burgeoning field of healthcare data analysis is continually confronted with the dual challenges of data privacy and the need for high-quality datasets with which to perform cutting edge data analysis and research. We aim to use a synthetic data generation pipeline to bridge the gap between these challenges by producing synthetic patient data capable of performing real research.
METHODS: We utilize GuardantINFORM, which includes anonymized genomic and structured payer claims data, to generate lung adenocarcinoma (LUAD) patient cohorts. We process this data into a tabular format before decomposing it into an internal sequential format that is able to leverage advanced transformer-based large language models. This innovation allows for the creation of synthetic datasets that not only retain the statistical properties of the original data but also preserve the privacy of underlying data points.
RESULTS: We show that there is a 0.993 R2 correlation between real and synthetic variable probabilities, demonstrating the robustness of the synthetic data in mimicking real-world data distributions. We then use a membership inference attack test, with a resulting accuracy of 49% or roughly a random guess, to underscore the effectiveness in preserving patient privacy while generating that data. Finally, we replicated a real study using the synthetic data by showing that LUAD patients with the KRAS G12C mutation were more susceptible to earlier loss of life in the face of a cancer diagnosis and that this problem compounded upon co-occurring STK11 mutations. This not only validates the practical utility of synthetic data but also highlights its potential in facilitating groundbreaking research.
CONCLUSIONS: We present a promising solution for the healthcare research community. It enables the sharing of privatized, synthetic data that retains the integrity and utility required for significant scientific discoveries, paving the way for more liberalized, responsible, and effective data sharing practices in healthcare research.
Conference/Value in Health Info
Value in Health, Volume 27, Issue 6, S1 (June 2024)
Acceptance Code
P53
Topic
Methodological & Statistical Research
Topic Subcategory
Artificial Intelligence, Machine Learning, Predictive Analytics
Disease
no-additional-disease-conditions-specialized-treatment-areas, Oncology