How Can Explainable Artificial Intelligence Accelerate the Systematic Literature Review Process?

OBJECTIVES: Systematic literature reviews (SLRs) are key evidence requirements for health technology agency decision-making. However, the exponential increase in published articles makes a thorough and practical literature review increasingly challenging. To help researchers conduct an SLR, we developed a machine learning (ML)-based pipeline to accelerate the title and abstract screening (TIABS) step. We assessed this ML-based TIABS using various human-labeled SLRs to ensure its reproducibility.

METHODS: Three human- performed SLRs spanning various therapeutic areas were used as training and validation data resources. Multiple 5-shot binary classifiers based on the Population-Intervention-Comparator-Outcome framework were trained for each SLR and used to build an explainable TIABS pipeline. Each abstract was assigned a predicted label after being passed through the binary classifiers for each SLR. Abstracts could be included or excluded and the reasons for exclusion were obtained concurrently. Only 5 data per class were used to train the 5-shot models.

RESULTS: The size of the labeled review datasets are 1626, 777 and 965 abstracts. With the minimal training data, the accuracy of the exclusion reasons predicted by the 5-shot classifiers ranged from 0.64 to 0.95, with an average accuracy of 0.80 across 3 different SLRs. The accuracy, recall, precision, and F-measure ranged between 0.81 and 0.85, 0.55 and 0.74, 0.66 and 0.78, 0.54 and 0.75, respectively.

CONCLUSIONS: The ‘5-shot classifier pipeline’ requires minimal training data while simultaneously speeding up the review process by providing explainability to the title and abstract screening prediction. However, further experimentation with an actual SLR is required to compare and evaluate the speed and accuracy of both manual and AI-assisted screening using this approach.