Presentation (CTI)

OBJECTIVES: The rapid adoption of evidence-driven decision-making in medicine, public health, and other fields has fueled a surge in evidence synthesis efforts, including systematic reviews (SRs), targeted literature reviews (TLRs), and evidence mapping. These processes, however, remain time-intensive and resource- demanding, requiring experts to extract and synthesize data from vast pools of literature. Recent advancements in Large Language Models (LLMs), such as OpenAI's GPT and Meta's LLaMA, offer transformative potential to address these challenges by automating key tasks. Yet, issues like hallucinations, lack of transparency, and inconsistent outputs hinder reliability and scalability.
METHODS: We explored the use of LLM to assist a research group with data extraction for a study involving 193 articles about digital health. We extracted 40 structured data elements, including demographic details, intervention methods, and outcomes. The LLM processed text, tables, and charts directly from the articles, generating responses to structured queries, which were then verified by the research group. For each data element, the LLM provided highlighted passages from the articles to direct researchers to relevant sections.
RESULTS: Productivity increased from an average of 80 extracted elements per hour for human-only extraction to 195 extracted elements per hour. While the accuracy of LLM-generated responses varied, the highlighted passages consistently directed researchers to relevant sections. Although not formally measured, this approach appeared to reduce user fatigue, allowing individuals to extend their work by 1-2 hours without breaks.
CONCLUSIONS: This multi-layered approach shows great potential to enhance evidence synthesis. LLM technologies combined with human oversight has demonstrated the ability to reduce time, costs and decision fatigue. Although still in early stages, this approach represents a scalable solution capable of accelerating evidence-based decision-making across diverse fields, including clinical research, public health policy, and education. Future advancements in LLM technology and workflow refinement may further solidify its role in transforming evidence synthesis.