Presentation (CTI)

OBJECTIVES: This study evaluated the efficacy of a Large Language Model (Google Gemini) in automatically drafting patient safety narratives (PSNs) by extracting data, aiming to enhance adverse event tracking efficiency.
METHODS: We engineered a retrieval-augmented generation (RAG) prompt to create a PSN template, aligned with International Conference on Harmonisation (ICH) guidelines for Clinical Study Report (CSR) structure. An Automatic Prompt Engineering (APE) method was then used to develop an extraction prompt. PSNs were generated from 30 published case reports of semaglutide-induced adverse events. Model performance was assessed across four narrative sections (Patient Demographics and Study Information, Relevant Medical History, Adverse Event/Serious Adverse Event Details, and Relevant Laboratory/Diagnostic Test Results), using three 1-10 scoring metrics per section.
RESULTS: Information retrieval and output generation averaged 10 seconds per case report, with an average input text length of 1025 words. The model achieved a general average score of 7.5 across all sections. However, the first section (Patient Demographics and Study Information) received the lowest scores, particularly for "Clarity of Study Drug Administration Dates/Group" (6.4) and "Completeness of Core Demographics" (7).
CONCLUSIONS: The model successfully extracted key information and generated concise PSNs efficiently. Nevertheless, continuous human oversight remains essential to ensure optimal data completeness and clarity.