OBJECTIVES: SR-CCEO should follow the same structured approach as SRs of effectiveness. Screening is one of the most resource-intensive stages and typically requires two independent reviewers to avoid errors. ISPOR Good Practices for Critical Appraisal of SR-CCEO lists two screening approaches to accelerate the process: single reviewer screening and text mining. However, their effectiveness is still debated. We explored whether an AI-assisted, semi-automated three-stage screening approach (AI-assisted single screening – AISS) improves efficiency in identifying the relevant studies for SR of economic evaluations and if it is a safe alternative to the traditional two-reviewers screening approach.

METHODS: We selected one SR-CCEO of the cost-effectiveness of pharmacological management for osteoarthritis for this study. We deduplicated all retrieved records using the machine learning model and uploaded them to Laser AI tool for three-stage screening: 1) calibration exercise – training phase for both screeners and AI model, 2) main screening phase – single screening by a junior reviewer supported by AI prioritization, and 3) quality assurance performed by a senior reviewer of only those records for which the decisions of the junior reviewer and the AI model disagreed. The primary endpoints of the study were the proportion of records or studies missed compared with the reference standard – the original SR. Secondary outcomes were: total time spent on screening, workload, and time saved in comparison with double screening.

RESULTS: We identified 4459 records and 215 were finally included for full text screening. Our approach found all of the 38 studies included in the original review. Title and abstract screening workload was reduced by 43%, whereas the estimated time saving was 19 hours and 16 minutes (total screening time 20h 57min).

CONCLUSIONS: Our results suggest that the AI-assisted single screening (AISS) approach might reduce human effort in SR-CCEOs. Further validation in a broader range of SR-CCEOs is ongoing.