Improving Literature Review Efficiency: Evaluating the Impact of the Opensource Asreview Tool on Time and Effort to Conduct Systematic and Targeted Reviews

OBJECTIVES: A systematic literature review (SLR) is a required health technology assessment tool to address evidence gaps, identify appropriate inputs for economic models, and to survey literature for use in indirect treatment comparisons. With the introduction of the European Joint Clinical Assessment in 2025, the timelines and expanded scope of required SLRs will be challenging to conduct using traditional methods. To improve and facilitate the SLR and targeted literature review (TLR) process, we tested AsReview, an open-source machine learning software to accelerate textual data screening to validate the title and abstract screening of two manually performed SLRs and one TLR.

METHODS: We utilized the AsReview tool to validate two SLRs and one TLR previously performed manually. Each review focused on a distinct therapeutic area, with search results ranging from 333 to 6045 records (332, 1113, and 6045 records, respectively). We evaluated the performance of the software by calculating two key metrics for each review: Work Saved over Sampling (WSS) and Average Time to Discover (ATD).

RESULTS: We recorded WSS values of 0.64, 0.71, and 0.60. These results indicate that AsReview screened only 36%, 29%, and 40% of the total datasets to detect all relevant records. ATD values were recorded as 15.76, 353.75 and 136.33 respectively, indicating the number of papers that needed to be screened to find the total relevant records. Taken together, these results suggest that AsReview significantly reduces the time taken to conduct literature review and title screening.

CONCLUSIONS: Our findings suggest that AsReview is an important tool used in the SLR and TLR validation process, and it can substantially diminish the time-intensive task for experienced reviewers conducting both SLRs and TLRs.