FORE SIGHTING THE ADOPTION OF ARTIFICIAL INTELLIGENCE-BASED DIAGNOSTIC APPLICATIONS IN ONCOLOGICAL IMAGING: A STRUCTURED EXPERT ELICITATION STUDY
Author(s)
Jamie Verwey, MSc1, Sandra Sülz, BSc, MSc, PhD2, Asabie Leliveld, MD3, Gonzalo Mosquera Rojas, MSc,3, Xinyi Wan, MSc,3, Jacob Visser, MD, MSc, PhD3, Maarten IJzerman, BSc, MSc, PhD4.
1Erasmus University Medical Center/Erasmus University Rotterdam, Rotterdam, Netherlands, 2Erasmus University Rotterdam, Rotterdam, Netherlands, 3Erasmus Medical Centre, Rotterdam, Netherlands, 4Erasmus School of Health Policy & Management, Rotterdam, Netherlands.
1Erasmus University Medical Center/Erasmus University Rotterdam, Rotterdam, Netherlands, 2Erasmus University Rotterdam, Rotterdam, Netherlands, 3Erasmus Medical Centre, Rotterdam, Netherlands, 4Erasmus School of Health Policy & Management, Rotterdam, Netherlands.
OBJECTIVES: Investment and development of radiological artificial intelligence (AI) is increasing rapidly, however many applications fail to reach clinical implementation. This study estimated the likelihood of clinical adoption for three radiological AI categories in oncological imaging and identified key determinants influencing AI adoption
METHODS: The Sheffield Elicitation Framework (SHELF) informed the elicitation and analysis process. Experts provided minimum, maximum, and most likely parameters of adoption likelihood to construct individual Program Evaluation and Review Technique (PERT) distributions. Monte Carlo simulation and kernels density estimations were used to generate pooled linear distributions. Individual certainty and agreement amongst experts was assessed using 80% Highest Density Intervals (HDIs). Calibration questions evaluated clinical expertise and patient contact to determine proximity to the intended adoption context. Additionally, experts ranked determinants influencing AI adoption.
RESULTS: Thirteen experts evaluated the three AI scenarios. Incidental pulmonary nodule detection (IPND), bone tumour diagnostics (BT), and non-invasive glioma subtyping (NIGS). IPND demonstrated the highest estimated likelihood of adoption (median 52.3%), followed by NIGS (34.3%) and BT (25.9%). Limited agreement was observed across scenarios, with HDI widths ranging from 55.3 to 63.3. Visual inspection indicated multimodal distributions, suggesting differing expectation profiles regarding AI adoption’s likelihood. Clinically embedded experts working in academic hospitals estimated lower likelihoods with narrower intervals. Conversely, higher likelihood estimates were associated with wider uncertainty intervals, indicating heterogeneity in both likelihood estimates and associated uncertainty, whereas strong agreement was observed in the ranking of adoption determinants.
CONCLUSIONS: The findings reflect substantial variation in AI adoption likelihood expectations, and the quantified uncertainty. These results reflect the inherent uncertainty of early assessment of complex healthcare technologies, but equally highlight the importance of involving diverse stakeholder groups early in the technology lifecycle. Improved methods for early value assessment of radiological AI applications are needed to better inform development, investment, and implementation decisions.
METHODS: The Sheffield Elicitation Framework (SHELF) informed the elicitation and analysis process. Experts provided minimum, maximum, and most likely parameters of adoption likelihood to construct individual Program Evaluation and Review Technique (PERT) distributions. Monte Carlo simulation and kernels density estimations were used to generate pooled linear distributions. Individual certainty and agreement amongst experts was assessed using 80% Highest Density Intervals (HDIs). Calibration questions evaluated clinical expertise and patient contact to determine proximity to the intended adoption context. Additionally, experts ranked determinants influencing AI adoption.
RESULTS: Thirteen experts evaluated the three AI scenarios. Incidental pulmonary nodule detection (IPND), bone tumour diagnostics (BT), and non-invasive glioma subtyping (NIGS). IPND demonstrated the highest estimated likelihood of adoption (median 52.3%), followed by NIGS (34.3%) and BT (25.9%). Limited agreement was observed across scenarios, with HDI widths ranging from 55.3 to 63.3. Visual inspection indicated multimodal distributions, suggesting differing expectation profiles regarding AI adoption’s likelihood. Clinically embedded experts working in academic hospitals estimated lower likelihoods with narrower intervals. Conversely, higher likelihood estimates were associated with wider uncertainty intervals, indicating heterogeneity in both likelihood estimates and associated uncertainty, whereas strong agreement was observed in the ranking of adoption determinants.
CONCLUSIONS: The findings reflect substantial variation in AI adoption likelihood expectations, and the quantified uncertainty. These results reflect the inherent uncertainty of early assessment of complex healthcare technologies, but equally highlight the importance of involving diverse stakeholder groups early in the technology lifecycle. Improved methods for early value assessment of radiological AI applications are needed to better inform development, investment, and implementation decisions.
Conference/Value in Health Info
2026-11, ISPOR Europe 2026, Vienna, Austria
Value in Health, Volume 29, Issue 12S
Code
HTA22
Topic
Health Service Delivery & Process of Care, Health Technology Assessment, Medical Technologies
Topic Subcategory
Value Frameworks & Dossier Format
Disease
No Additional Disease & Conditions/Specialized Treatment Areas, Oncology