Evaluation of Natural Language Processing (NLP) on Electronic Medical Records: A Proof of Concept on Chronic Graft Versus Host Disease (cGVHD) in France
Author(s)
Colas S1, Gilles V1, Chantepie S2, Daguindau E3, Huynh A4, Loschi M5, Robin M6, Raus N7, Requillard C8, Chuttoo L8, Poplu A8, Allali N1, Kiprijanovski D1, Leouay F8, Jeanbat V9, Kirion J9, Cottin J9, Buchbinder N10, François S11, Villate A12, Bouee S13
1Sanofi, Paris, France, 2CHU de Caen, Caen, France, 3CHU de Besançon, Besançon, France, 4Oncopole of Toulouse, Toulouse, France, 5CHU de Nice, Nice, France, 6APHP, Paris, France, 7SFGM-TC, Paris, France, 8LIFEN, Paris, France, 9CEMKA, Bourg-la-Reine, France, 10CHU de Rouen, Rouen, France, 11CHU Angers, Angers, France, 12CHU de Tours, Tours, France, 13CEMKA, BOURG LA REINE, France
OBJECTIVES: Manual chart review used to generate real-world data can be long and expensive. We used an innovative approach based on an artificial intelligence natural language processing (AI-NLP) method to structure data into a cohort of patients affected by chronic Graft Versus Host Disease (cGVHD) following hematopoietic stem-cell transplantation (alloHSCT).
We aimed to demonstrate this approach effectiveness in terms of data quality, along with related time saving.METHODS: The NLP scanned all electronic medical records, identified eligible patients and used relevant information to derive and infer the variables.
To ensure completeness and accuracy of the NLP derived data, a three-step quality check was performed: 1) review of variables derivation by clinical research associate (CRA) 2) their comparison with the European Bone Marrow Transplant (EBMT) registry 3) their validation by investigator experts.RESULTS: From all alloHSCT patients’ medical files (N=306), 20 met the selection criteria, whose data were extracted with NLP. This comprehensive process took 30 hours by a data scientist and 0.5 hour of investigator expert time per center.
The completeness of data derived by NLP varied according to nature of the variables: sociodemographics and cGVHD treatments (100%), graft characteristics (>75%), alcoholism (20%), mostly depending on the information documented in the medical records. The review by CRA did not lead to discordance except for the cGVHD treatments, for which some adjustments were needed. The quality checked against EBMT registry showed high accuracy of the NLP derived data for all 20 patients. Concordance rates were: stem cell source, donor gender and type (100%), alloHSCT indication (94%), alloHSCT date (89%), CMV positivity (87%), conditioning and prophylactic treatment of cGVHD (78%). Validation by investigator experts is still ongoing.CONCLUSIONS: This innovative AI-NLP process allows to generate a comprehensive, robust and accurate structured dataset, from scattered information of all eligible patients, in medical records, with minimal human intervention.
Conference/Value in Health Info
Value in Health, Volume 27, Issue 12, S2 (December 2024)
Acceptance Code
P47
Topic
Methodological & Statistical Research, Real World Data & Information Systems, Study Approaches
Topic Subcategory
Artificial Intelligence, Machine Learning, Predictive Analytics, Data Protection, Integrity, & Quality Assurance, Electronic Medical & Health Records
Disease
Oncology, systemic-disorders-conditions-anesthesia-auto-immune-disorders-n-e-c--hematological-disorders-non-oncologic-pain