Recommendations on the Use of Artificial Intelligence and Machine Learning in Systematic Literature Reviews Submitted as Part of the Evidence Package in Health Technology Assessment

OBJECTIVES:

The benefits of a health intervention are commonly assessed through a process known as a health technology assessment (HTA) whereby the interventions benefits are systematically assessed. This requires the searching, selection, extraction, and critical assessment of all available clinical and economic published literature. This may result in a systematic literature review (SLR) with many published studies having to be assessed within a limited timeframe. Advanced analytic techniques such as artificial intelligence (AI) and machine learning (ML) afford reviewers the opportunity for increased efficiency. We aimed to review available guidance on the use of AI/ML in HTA-based SLRs.

METHODS:

We reviewed relevant documents including methodological guidance for the following HTA bodies: NICE (England), HAS (France), G-BA (Germany), NCPE (Ireland), SMC (Scotland), TLV (Sweden), CADTH (Canada) and PBAC (Australia) to understand their acceptance of AI and ML in SLRs submitted as part of HTA. We additionally reviewed guidance from Cochrane as this is a respected body in the field of SLRs with HTA bodies commonly aligning to their recommendations.

RESULTS:

Currently, no explicit reference is made regarding use or acceptance of AI and ML in any available document in relation to the conduct of SLRs for HTA. NICE and NCPE expect two reviewers to be involved in the SLR but do not state whether AI/ML is suitable as one of these; SMC refers readers to NICE methodologies. Significantly, Cochrane, which is often referred to and referenced by HTA bodies for guidance on best practise in SLRs, is undertaking workstreams to understand how to best exploit AI/ML in SLRs to improve efficiency and output quality.

CONCLUSIONS:

Although AI/ML is not currently acknowledged as best practise in HTA-based SLRs, there are strong indications that the field is moving in this direction and further guidance from HTA bodies is required.