CAN PUBMED ALONE CAPTURE THE EVIDENCE BASE FOR TARGETED LITERATURE REVIEWS? A CASE STUDY
Author(s)
Priscilla Wittkopf, PhD1, Rajat Goel, MPharm2, Kassandra Schaible3, Caroline von Wilamowitz-Moellendorff, PhD1;
1Thermo Fisher Scientific, London, United Kingdom, 2Thermo Fisher Scientific, Mumbai, India, 3Thermo Fisher Scientific, Pittsburgh, PA, USA
1Thermo Fisher Scientific, London, United Kingdom, 2Thermo Fisher Scientific, Mumbai, India, 3Thermo Fisher Scientific, Pittsburgh, PA, USA
OBJECTIVES: Targeted literature reviews (TLRs) commonly combine MEDLINE and Embase, to optimise study retrieval. Embase often yields larger results sets due to broader indexing, and differences may be amplified when pragmatic filters are applied. To our knowledge, PubMed is the only database which allows programmatic access and does not impose AI-specific restrictions on the use of external models for analysis of its records. The implications of PubMed-only strategies for AI-assisted TLRs remain unclear. We aimed to compare search results using PubMed-only with those identified using Medline plus Embase in the context of two AI-assisted TLRs.
METHODS: Original searches were conducted in MEDLINE plus Embase (via Ovid) in October-November 2024. Search syntax was adapted for PubMed and rerun in January 2025 using the original date limits. Identical eligibility criteria and AI-assisted screening workflow (Nested Knowledge) were used across approaches. Outcomes included the overlap between approaches and studies uniquely identified through each database.
RESULTS: For TLR 1 (cost-effectiveness; validated economic filter), PubMed-only retrieved 2,285 records from versus 2,426 articles from the combined MEDLINE + Embase search, with 92.3% overlap of relevant included records. TLR 2 (observational studies; pragmatic design filter), PubMed-only retrieved 937 records versus 2,212 from combined search, with 80.0% overlap of relevant included records. Records not captured by PubMed-only were uniquely identified via Embase, consistent with broader database coverage and indexing.
CONCLUSIONS: Across two AI-assisted TLRs, PubMed-only and combined Medline plus Embase searches yield overlapping evidence bases; however, gaps were larger when a pragmatic observational study design filter was used. AI-assisted screening appears to be a cost-efficient approach and performed consistently across differing record pools, but the final included evidence will depend on where searches are run (database coverage) and how they are run (choice and validation of filters). These findings highlight the importance of understanding database-specific coverage when designing AI-assisted TLR search strategies.
METHODS: Original searches were conducted in MEDLINE plus Embase (via Ovid) in October-November 2024. Search syntax was adapted for PubMed and rerun in January 2025 using the original date limits. Identical eligibility criteria and AI-assisted screening workflow (Nested Knowledge) were used across approaches. Outcomes included the overlap between approaches and studies uniquely identified through each database.
RESULTS: For TLR 1 (cost-effectiveness; validated economic filter), PubMed-only retrieved 2,285 records from versus 2,426 articles from the combined MEDLINE + Embase search, with 92.3% overlap of relevant included records. TLR 2 (observational studies; pragmatic design filter), PubMed-only retrieved 937 records versus 2,212 from combined search, with 80.0% overlap of relevant included records. Records not captured by PubMed-only were uniquely identified via Embase, consistent with broader database coverage and indexing.
CONCLUSIONS: Across two AI-assisted TLRs, PubMed-only and combined Medline plus Embase searches yield overlapping evidence bases; however, gaps were larger when a pragmatic observational study design filter was used. AI-assisted screening appears to be a cost-efficient approach and performed consistently across differing record pools, but the final included evidence will depend on where searches are run (database coverage) and how they are run (choice and validation of filters). These findings highlight the importance of understanding database-specific coverage when designing AI-assisted TLR search strategies.
Conference/Value in Health Info
2026-05, ISPOR 2026, Philadelphia, PA, USA
Value in Health, Volume 29, Issue S6
Code
MSR237
Topic
Methodological & Statistical Research
Topic Subcategory
Artificial Intelligence, Machine Learning, Predictive Analytics
Disease
No Additional Disease & Conditions/Specialized Treatment Areas