OBJECTIVES: Artificial intelligence (AI)-based technologies are rapidly developing with the potential to improve healthcare quality at reduced cost. However, few examples exist of successfully deployed AI-technologies that have been adequately assessed. Therefore, the objective of this research is to: (i) examine the usability of the EUnetHTA Core Model for assessment of AI-based health-technologies (ii) validate new topics relevant for AI-uptake, and (iii) take the first step in developing a HTA-framework for AI-based technologies.

METHODS: A web-based expert Delphi-survey was conducted to identify critical evidence required to inform and support the decision-making process when implementing AI-based technologies. The survey comprised questions on the importance of including information from the 9 EUnetHTA Core Model domains used for the assessment of health technologies and additional topics related to the implementation of AI-technologies, identified through literature reviews. The survey included experts from the following categories: Clinician/researcher; HTA; Data-programmer/engineer; Ethicist/Bioethicist; Patients/advocates; Health-economist; Health-policy; Legal.

RESULTS: Out of 87 experts invited 53 responded to the first Delphi round (response rate = 61%). The Core Model domains most often considered “Critical to include” were: Clinical effectiveness (82%) and Ethical analysis (81%), whereas domains less considered “Critical to include” were: Organizational aspects (59%) and Social aspects (63%). Of the new topics identified, “Trustworthiness” and “Human agency and oversight” were most often considered critical to include (86% and 84%) whereas “Environmental sustainability“ and “Manufacture of technology” were less often considered critical to include (42% and 40%).

CONCLUSIONS: The results from this study uncovers key gaps in the Core Model to support the decision-making process when implementing AI-based health-technologies as well as identifies valuable information on relevant assessment aspects for AI. The results will form the basis for the development of a framework to assist decision-makers in assessing AI-based technologies in a holistic manner for a responsible deployment.