Presentation (CTI)

OBJECTIVES: Systematic literature review (SLR) data extraction is resource-intensive within health technology assessment (HTA), and subject to inter-reviewer variability. Recent guidance from the National Institute for Health and Care Excellence (NICE) recognises the potential role of artificial intelligence (AI) in evidence generation, while emphasising the need for transparent, justified, and validated use with appropriate human oversight. This study evaluated the accuracy and efficiency of an automated AI-assisted data extraction tool implemented within a NICE-aligned workflow incorporating human quality control (QC).
METHODS: Thirty journal articles were selected across clinical, economic, and quality of life topics, with 6-17 predefined data points extracted per study, including study design, objectives, population characteristics, and outcomes. Extractions were performed using the AI-based tool, followed by human QC to ensure alignment with HTA standards. Accuracy was defined as the proportion of data points extracted with sufficient precision such that human re-extraction was not required. Time required for AI-assisted and dual-human-only extraction was calculated for an identical theoretical dataset and compared to estimate time savings.
RESULTS: Across 30 publications and 319 data points, the overall accuracy of AI-assisted extractions was 86%. Most discrepancies identified during human QC required only minor content or formatting adjustments. By replacing manual extraction, the AI-assisted workflow has the potential to reduce overall extraction task time by 30-40%, even with additional time allocated to QC.
CONCLUSIONS: Within a NICE-aligned SLR workflow incorporating human QC, AI-assisted data extraction demonstrated high accuracy alongside reduced resource requirements. Beyond efficiency gains, the structured approach may support more standardised extraction while maintaining methodological rigor expected in HTA submissions. These findings suggest AI can be integrated into SLR workflows within HTA in line with emerging guidance, supporting both efficiency and process robustness, with broader application across SLR workflows likely to further inform its impact on quality and efficiency.