Applications, Challenges, and Limitations of AI Use in Healthcare: An Analysis of AI Responses for Drug Information

OBJECTIVES: Health professionals are increasingly interested in the potential applications of artificial intelligence (AI) in routine healthcare. The primary focus of the discussions lie on accuracy, validity and reliability of the AI-generated information. This study delves into the multifaceted relationship between AI and healthcare scrutinizing the validity and reliability of AI-generated drug information.

METHODS: The study investigates responses of ChatGPT to questions on Drug-Drug Interactions (DDIs). A comprehensive, systematic literature search on the use of AI in routine healthcare – especially concerning the use of pharmaceuticals – was performed. Exemplary fields operationalized by 40 standardized questions on DDIs where defined. Multiple responses of ChatGPT were investigated and accuracy and reliability compared to clinical guidelines as gold standard were assessed.

RESULTS: The probability of obtaining a correct interaction was found to be 75%. For 80% of the correct AI information examples ChatGPT provided the correct information on the level of interaction and scientific evidence base, too. Conversely, the probability of encountering interactions that were both incorrect and insufficient was 25%. All answers based on generative AI were consistent after multiple repetitions. ChatGPT showed signs of consecutive incorrect answers, similar to the phenomenon known as “hallucinations”, rising questions about the ability of providing accurate information over long periods of inquiries, and whether these imperfections are still inheritable to more sophisticated and specialized systems.

CONCLUSIONS: AI-generated answers to potentially clinically relevant question on DDIs are available and widely used. It is convenient, fast and practically free of additional costs but the answers of generative AI show some lack of validity although a satisfying reliability. For a relevant number of cases ChatGPT could not provide evidence-based description of the pharmaceutical mechanism. An error rate of 25% and presence of the “hallucinations” phenomenon implicates that healthcare professionals should not fully rely on ChatGPT as a source of information.