OBJECTIVES:

The growing number of health economic papers increases the workload of screening papers during systematic literature reviews. Recently, artificial intelligence (AI)-powered tools have been introduced, aiming at reducing the time needed for title and abstract screening. The open source tool ASReview ranks papers based on their likelihood of being relevant using the labels ‘relevant’ and ‘irrelevant’, and iteratively updates this ranking following each single assessment performed by reviewers. This study aimed to assess the accuracy and efficiency of the AI-tool within health economics.

METHODS:

A sample of 4,994 papers focussing on health economic evaluations of early detection strategies for cardiovascular disease was used for this study. Manual screening of this sample resulted in 117 studies included for full-text screening (FT) and 50 for data extraction (DE). The AI-tool requires reviewers to provide prior knowledge by feeding the algorithm one relevant and one irrelevant paper. Using the dynamic ranking provided by the AI-tool and screening 5%, 10%, and 20% of the highest-ranked papers, the proportion of papers manually identified for FT and DE papers was determined. Robustness of the AI-tool was determined by varying prior knowledge.

RESULTS:

After screening 5%, 10% and 20% of the papers highest-ranked by the AI-tool, respectively 55.4%, 74.4%, and 84.6% of FT papers were found and respectively 96%, 100%, and 100% of DE papers were found. Identify all manually identified FT and DE papers using the AI-tool required assessing respectively 88.1% and 6.5% of all papers. When varying the prior knowledge (initial provided relevant and irrelevant paper), all DE papers were found in the top 6.2% to 9.1% of ranked papers.

CONCLUSIONS:

All DE papers identified manually were found by the AI-tool after screening less than 10% of the full sample. Using the AI-tool could substantially reduce the time required for screening in systematic reviews of health economic papers.