Transforming Query and Data Retrieval Systems With the Advanced Power of GPT-4o: Generative AI at the Forefront of Extracting Data

Author(s)

Kaur R1, Soni V2, Waddell N3, Pandey S1, Kaur G1, Singh B4
1Pharmacoevidence, Mohali, India, 2Pharmacoevidence, Mohali, PB, India, 3Pharmacoevidence, Edinburgh, UK, 4Pharmacoevidence, SAS Nagar Mohali, PB, India

OBJECTIVES: Vast amounts of data are generated daily, and efficient query and data retrieval systems are essential for deriving actionable insights. Traditional methods often struggle with the complexity and volume of data. The objective of the study was to automate the precise data extraction from research articles by developing a query interface.

METHODS: This dynamic interface was developed to automate the precise data extraction from research articles using GPT-4o and Python to provide an interactive user experience using Streamlit. The uploaded files were divided into smaller and more manageable chunks. These chunks were encoded using OpenAI Embeddings and indexed using FAISS to maximize the searches based on similarity. The interface allows users to enter precise queries for data retrieval from documents. Based on user queries, GPT-4o uses the most relevant chunks to provide comprehensive, contextually relevant responses, increasing the efficiency and efficacy of information retrieval.

RESULTS: The interface functionality was evaluated using a set of 56 prompts across four distinct research publications. Domain experts extensively tested the system's responses, finding satisfactory performance with correct data retrieval for 50 of 56 prompts, though inconsistencies in 6 prompts indicated a need for further optimization.

CONCLUSIONS: This AI-powered interface, using GPT-4o and OpenAI embeddings, achieved remarkable accuracy in retrieving information from research articles, as validated by experts. It provides quick access to crucial insights, potentially speeding up research and facilitating informed decision-making in various domains related to patient health and healthcare economics. This technique can potentially improve efficiency, minimize time, and optimize resource requirements in HEOR information retrieval.

Conference/Value in Health Info

2024-11, ISPOR Europe 2024, Barcelona, Spain

Value in Health, Volume 27, Issue 12, S2 (December 2024)

Code

MSR167

Topic

Methodological & Statistical Research

Topic Subcategory

Artificial Intelligence, Machine Learning, Predictive Analytics

Disease

No Additional Disease & Conditions/Specialized Treatment Areas

Explore Related HEOR by Topic


Your browser is out-of-date

ISPOR recommends that you update your browser for more security, speed and the best experience on ispor.org. Update my browser now

×