OBJECTIVES: Despite the vast selection of available radiology artificial intelligence (AI) tools, relatively few are used in clinical practice. To improve the use and value of radiology AI tools, early health technology assessment (eHTA) should be considered. However, a comprehensive eHTA for radiology AI tools is currently missing. The aim of this paper is to introduce and evaluate an eHTA procedure for AI in radiology.

METHODS: The developed eHTA procedure addresses the aspects of comprehensiveness, value weighting and performance prediction specifically for radiology AI tools. For this, a methodological framework consisting of the HTA Core Model domains was supplemented with the AI-specific guiding principles of the FUTURE-AI guideline to ensure comprehensiveness of the eHTA procedure. Based on the ISPOR recommendations for Multiple Criteria Decision Analysis, an analytical hierarchy process is described which includes a literature review, stakeholder elicitation by interviews, a focus group session, and a questionnaire to determine value weighting and performance predictions of the evaluated radiology AI tool. For development and evaluation of the eHTA procedure, a radiology AI tool being developed for management of incidental pulmonary embolisms (iPE) was used as pilot study.

RESULTS: Preliminary results have shown that the eHTA procedure provides the opportunity to critical evaluate the iPE AI tool during development, thereby improve the models’ design to fit clinical practice. For example, it identified previously unknown effects that the AI tool will have on patients, resulting in alterations of the tools objectives. Furthermore, the eHTA procedure forms the starting point for an iterative process updating an AI tool’s value proposition through the lifecycle.

CONCLUSIONS: The developed methodology is a rapid, systematic process which can provide a practical, comprehensive eHTA for AI in radiology. For future research, a health technology assessment will be performed to evaluate the eHTA predictive capacity.