OBJECTIVES: The presence of administrative tasks significantly contributes to physician burnout, impacting healthcare costs and compromising quality. Therefore, the development of tools to reduce clerical burden can optimize healthcare pathways. This study aimed to construct an automated report generator for a cardiology service using a language model and evaluate its performance using algorithmic and human metrics.

METHODS: A Guiding Multiple Choice and Field Completing System (MCFCS) was developed with Python 3.9. It integrated patient identification, gender, age, blood pressure, breath and heart sounds, and ECG/echocardiogram results. Variables were compiled using the Chat GPT API (text-davinci-003) to generate plain text reports with varying temperature values. Word count and generation time were measured. Evaluation involved cardiologist assessment and models such as BERT, BIO CLINICAL BERT, ROBERTA, and Chat GPT, measuring time and assigning scores (1-10). Additional information inclusion was also evaluated (score: 0-3).

RESULTS: Reports generated using the Chat GPT API demonstrated successful outcomes, with an average generation time of 8.7 seconds and an average word count of 70. Evaluation by cardiologists required an average of 22 seconds, yielding an average score of 8. Chat GPT performed closest to cardiologists, with an average score of 7.45 and an evaluation time of 0.13 seconds. In contrast, other models (BERT, BIO CLINICAL BERT, ROBERTA) took longer (above 1.6 seconds on average) and achieved inferior performance with scores of 3.2, 2.5, and 2.5, respectively. Notably, the highest score assigned by physicians was attained using a temperature value of 0.5 in the Chat GPT API.

CONCLUSIONS: The implementation of an automated reporting application enables the rapid generation of high-quality medical reports. Further advancements and refinements hold significant potential for alleviating the burdensome administrative tasks faced by physicians, thus mitigating burnout and optimizing overall healthcare system efficiency.