Presentation (CTI)

OBJECTIVES: Literature reviews are resource-intensive, with time and cost scaling in proportion to corpus size. Exploratory literature reviews (ELRs), often using large language models (LLMs), may precede targeted literature reviews (TLRs) to understand the evidence landscape, identify key articles and inform whether a full TLR is warranted. However, screening in ELRs still requires evaluation of all articles, and LLM use usually incurs scaling processing costs and time for iterative prompt development. We developed a rapid, zero-data-processing-cost workflow for ELRs using a biomedical transformer fine-tuning approach, offering an alternative to LLMs for high-volume screening. We evaluated the approach through a use-case in rare kidney disease.
METHODS: MEDLINE, Cochrane and Embase searches identified 2793 abstracts. Two independent reviewers labelled a random sample (n=500) of abstracts for binary relevance to the research question, with an independent reviewer resolving discrepancies. Grid search was used to identify the best base model and hyperparameters. The best-performing model was applied to the full corpus, and human experts validated the 50 top-scoring articles.
RESULTS: The best-performing model was PubMedBERT (learning rate=2e-05, decay=0.1, batch=4). Validation on a 20% subset of labelled articles indicated Area Under Receiver Operating Characteristic Curve (AUC)=0.96, F1=0.73 and average precision=0.82. Human review confirmed all top-50 unseen abstracts were relevant (precision@50=1). The model estimated an inclusion prevalence of 17.98% across unseen abstracts (n=2293); validation projection estimated that 80% of inclusions would appear in the top 367 abstracts and 90% in the top 436.
CONCLUSIONS: Our approach facilitates rapid prioritization of key articles and estimation of total literature relevance, helping determine whether a full TLR is warranted. It is well-suited to broad research questions and large corpora. It does not require data egress or inference costs, making it an alternative to LLMs where these factors may be prohibitive.