AI in Evidence Synthesis: Have We Reached the Promised Land or Are We Still Wandering the Desert?
Author(s)
Maciej Grys, PhD1, Kevin Kallmes, BS, MA, JD2, Jeff Johnson, BA3, Roman Casciano, MS4.
1Certara, Cracow, Poland, 2Nested Knowledge, St. Paul, MN, USA, 3Nested Knowledge, Corcoran, MN, USA, 4Certara USA Inc., Mamaroneck, NY, USA.
1Certara, Cracow, Poland, 2Nested Knowledge, St. Paul, MN, USA, 3Nested Knowledge, Corcoran, MN, USA, 4Certara USA Inc., Mamaroneck, NY, USA.
OBJECTIVES: Artificial Intelligence (AI) has transformed long-standing paradigms in the conduct of literature reviews (LR). This study evaluates the capabilities of a commercially available AI-enhanced LR tool to automate key elements of the review process.
METHODS: Following a direct comparative evaluation of four AI-assisted LR tools published in May 2025, the top-performing platform was adopted as our organization’s primary LR solution. In this study, we assessed its performance in abstract screening, data extraction, critical appraisal and report preparation, across multiple commercial LR projects.
RESULTS: Significant AI updates were implemented over the past year in the AI LR tool, AutoLit (Nested Knowledge), and tested in real-world projects. The tool enables automated literature searches by inputting a research question into “Smart Search,” eliminating the need for manual search strategy development. An AI exploration feature, which derives Populations, Interventions/Comparators, and Outcomes (PICOs) from abstracts, facilitated rapid screening and proved highly effective. As a second reviewer (“Robot Screener”), AI notably reduced screening time by 50% in large-scale projects (>4,500 abstracts). The most impactful feature was the automated extraction of text and tables from publications via Adaptive Smart Tags. PICOs and other key data were accurately extracted into user-defined templates. The tool also demonstrated flexibility in off-label applications: automated Critical Appraisal using RoB2 and JBI tools, and the generation of result summaries all showed high accuracy, outperforming human reviewers in some instances.
CONCLUSIONS: The adoption of an AI-enhanced LR tool significantly improved the efficiency and quality of literature reviews within our company. While the full potential of AI in LR has yet to be realized, current advancements have already demonstrated value. This progress may accelerate patient access to safe and effective medical technologies.
METHODS: Following a direct comparative evaluation of four AI-assisted LR tools published in May 2025, the top-performing platform was adopted as our organization’s primary LR solution. In this study, we assessed its performance in abstract screening, data extraction, critical appraisal and report preparation, across multiple commercial LR projects.
RESULTS: Significant AI updates were implemented over the past year in the AI LR tool, AutoLit (Nested Knowledge), and tested in real-world projects. The tool enables automated literature searches by inputting a research question into “Smart Search,” eliminating the need for manual search strategy development. An AI exploration feature, which derives Populations, Interventions/Comparators, and Outcomes (PICOs) from abstracts, facilitated rapid screening and proved highly effective. As a second reviewer (“Robot Screener”), AI notably reduced screening time by 50% in large-scale projects (>4,500 abstracts). The most impactful feature was the automated extraction of text and tables from publications via Adaptive Smart Tags. PICOs and other key data were accurately extracted into user-defined templates. The tool also demonstrated flexibility in off-label applications: automated Critical Appraisal using RoB2 and JBI tools, and the generation of result summaries all showed high accuracy, outperforming human reviewers in some instances.
CONCLUSIONS: The adoption of an AI-enhanced LR tool significantly improved the efficiency and quality of literature reviews within our company. While the full potential of AI in LR has yet to be realized, current advancements have already demonstrated value. This progress may accelerate patient access to safe and effective medical technologies.
Conference/Value in Health Info
2025-11, ISPOR Europe 2025, Glasgow, Scotland
Value in Health, Volume 28, Issue S2
Code
HTA29
Topic
Clinical Outcomes, Health Technology Assessment, Methodological & Statistical Research
Topic Subcategory
Systems & Structure, Value Frameworks & Dossier Format
Disease
No Additional Disease & Conditions/Specialized Treatment Areas