Abstract

Objectives

This systematic review assessed the scope, reporting quality, and methodological risk of bias of health economic evaluations (HEEs) of medical artificial intelligence (AI) technologies, alongside the technological maturity of the AI systems assessed.

Methods

Following Preferred Reporting Items for Systematic Reviews and Meta-Analyses 2020 guidelines, 6 databases were searched through April 2025 for studies reporting economic outcomes of AI applications in healthcare. Reporting quality was evaluated using the AI-specific update of the Consolidated Health Economic Evaluation Reporting Standards checklist, methodological risk of bias using the ECOBIAS framework, and AI maturity using the Clinical Machine Learning Readiness Level (1-9). Inclusion of implementation and operational costs was examined, as well as their association with AI maturity.

Results

A total of 117 studies were included, with most published after 2021. Reporting quality was generally suboptimal, and ECOBIAS assessments highlight recurring risks of bias, particularly regarding incomplete cost inclusion, limited data transparency, inadequate uncertainty analysis, and insufficient model validation. Most studies evaluated AI tools at early development stages (63% at Clinical Machine Learning Readiness Level 4-5), with limited real-world validation. Although the majority of studies reported cost savings or cost-effectiveness, key cost categories were frequently omitted: only 28% included implementation costs, and 57% reported operational costs.

Conclusions

Despite frequent claims of economic benefit, current HEEs of medical AI are constrained by limited reporting quality, risk of bias, and evaluations of immature technologies. Future HEEs should explicitly report technological maturity, incorporate full cost components, and use rigorous methods. Robust evaluations conducted at higher readiness levels are also needed to generate reliable evidence for policy making, reimbursement decisions, and responsible implementation.