Enhancing Targeted Literature Reviews (TLRs) with Artificial Intelligence (AI): A Methodological Approach for Conducting Efficient Targeted Searches
Author(s)
Caroline von Wilamowitz-Moellendorff, PhD1, Rajat Goel, MPharm2, Priscilla Wittkopf, PhD1, Sarah Ronnebaum, PhD3.
1Thermo Fisher Scientific, London, United Kingdom, 2Thermo Fisher Scientific, Mumbai, India, 3Thermo Fischer Scientific, Wilmington, NC, USA.
1Thermo Fisher Scientific, London, United Kingdom, 2Thermo Fisher Scientific, Mumbai, India, 3Thermo Fischer Scientific, Wilmington, NC, USA.
OBJECTIVES: Healthcare decision-makers often rely upon TLRs to rapidly identify, assemble, and synthesize available evidence. AI tools are currently being explored as tools to improve the efficiency and quality of TLRs. This study presents a novel method utilizing AI capabilities within the Nested Knowledge software to conduct robust TLRs across various research topics.
METHODS: Our approach involves training an AI model using known citations of interest. The model then scans the captured abstracts and assigns an advancement probability score to each, ranging from 0 (irrelevant) to 1 (highly relevant). Abstracts with a score of 0.8 or higher are screened by a single reviewer. Those meeting the selection criteria are reviewed at the full-text level and included if relevant.
RESULTS: The method was applied to five different TLRs covering clinical effectiveness, surrogate endpoint analysis, epidemiology, cost-effectiveness, and prognostic factors for disease progression. Across all reviews, less than 10% of abstracts required screening. Across TLRs recall range was 0.88-0.97, accuracy 0.73-0.94, and precision 0.43-0.73, indicating that the model uses a conservative approach to determining which citations should be included, resulting in effective prioritization by the tool. Abstract screening time was reduced by approximately 90%, while ensuring that the most relevant studies were identified and included in the reviews; this was a consistent benefit seen across a range of topics. Although not all articles were screened, the research questions were adequately addressed in all TLRs, demonstrating the robustness and reliability of the method.
CONCLUSIONS: AI-assisted screening in TLRs ensures efficient identification of critical evidence. Inclusion of grey literature, human validation of screening decisions, and thoughtful data interpretation and reporting is essential to support accurate evidence synthesis. Our findings suggest that AI can be a valuable tool in evidence synthesis, particularly in contexts where methodological flexibility is permissible.
METHODS: Our approach involves training an AI model using known citations of interest. The model then scans the captured abstracts and assigns an advancement probability score to each, ranging from 0 (irrelevant) to 1 (highly relevant). Abstracts with a score of 0.8 or higher are screened by a single reviewer. Those meeting the selection criteria are reviewed at the full-text level and included if relevant.
RESULTS: The method was applied to five different TLRs covering clinical effectiveness, surrogate endpoint analysis, epidemiology, cost-effectiveness, and prognostic factors for disease progression. Across all reviews, less than 10% of abstracts required screening. Across TLRs recall range was 0.88-0.97, accuracy 0.73-0.94, and precision 0.43-0.73, indicating that the model uses a conservative approach to determining which citations should be included, resulting in effective prioritization by the tool. Abstract screening time was reduced by approximately 90%, while ensuring that the most relevant studies were identified and included in the reviews; this was a consistent benefit seen across a range of topics. Although not all articles were screened, the research questions were adequately addressed in all TLRs, demonstrating the robustness and reliability of the method.
CONCLUSIONS: AI-assisted screening in TLRs ensures efficient identification of critical evidence. Inclusion of grey literature, human validation of screening decisions, and thoughtful data interpretation and reporting is essential to support accurate evidence synthesis. Our findings suggest that AI can be a valuable tool in evidence synthesis, particularly in contexts where methodological flexibility is permissible.
Conference/Value in Health Info
2025-11, ISPOR Europe 2025, Glasgow, Scotland
Value in Health, Volume 28, Issue S2
Code
SA38
Topic
Study Approaches
Topic Subcategory
Literature Review & Synthesis
Disease
No Additional Disease & Conditions/Specialized Treatment Areas