OBJECTIVES: Evidence synthesis is necessary to make informed decisions regarding patients and health policies. Literature reviews are used by the HEOR organizations to summarize all available evidence for answering specific clinical question. Medical literature is growing so rapidly that researchers have become overburdened with literature searches. Using artificial intelligence and machine learning (AI/ML) in semi-automation of the conventional manual literature review process could expedite the literature review process. We explored the applicability and validated the AI review tool of the Distiller SR platform (Evidence Partners Inc. Ottawa, Canada) for title/abstract screening.

METHODS: AI review tool was used as a reviewer in the single review screening of the targeted literature review. The tool trained itself based on the screening set (screened by a human reviewer; 587 previously screened records) and assigned scores between “0” (probability of high exclusion) and “1” (probability of high inclusion) to new unscreened references. The cut-off scores defined were ≤0.4 as exclude and ≥0.7 as include. To validate this approach, single reviewed references were cross-checked by a human reviewer.

RESULTS: AI review tool screened 118 references. Approximately 40.6% of the references required human intervention; 28.8% of references with cut-off scores between 0.4 to 0.7 and 11.8% of false excluded references. The manual screening of 118 references by a human reviewer takes around 3.0-3.5 hours, whereas this time was drastically reduced to 2 minutes by AI review tool. Further, the human reviewer took around 2 hours to resolve false exclusions and take decisions for unclear references.

CONCLUSIONS: AI review tool expedites the screening process with 40-50% time reduction with 90% accuracy. However, because of the 12% erroneous exclusions, AI/ML functionality cannot be utilized in systematic literature review conducted for regulatory submission or reimbursement dossiers.