NICE'S APPROACH TO TRIAGING AI USE CASES TO EXPLORE POTENTIAL USE FOR GUIDANCE PRODUCTION TASKS
Author(s)
Ahmed Yosef, BSc MSc1, Michael John Merchant, PhD2, Raphael Sonabend-Friend3, Stephen Duffield, PhD, MD4, Pall Jonsson, BSc, PhD5.
1NICE, London, United Kingdom, 2NICE, Manchester, United Kingdom, 3NICE, United Kingdom, 4NICE, Liverpool, United Kingdom, 5National Institute for Health and Care Excellence (NICE), Manchester, United Kingdom.
1NICE, London, United Kingdom, 2NICE, Manchester, United Kingdom, 3NICE, United Kingdom, 4NICE, Liverpool, United Kingdom, 5National Institute for Health and Care Excellence (NICE), Manchester, United Kingdom.
OBJECTIVES: To develop a triage tool for the National Institute for Health and Care Excellence (NICE) to support pragmatic, resource-sensitive prioritisation and testing of artificial intelligence (AI) use cases.
METHODS: A NICE HTA Innovation Laboratory (HTA Lab) project, “Future of HTA with AI”, examined the potential role of AI in supporting and evolving health technology assessment (HTA) methods. The project included a landscape review, external stakeholder workshops, and internal consultation. As part of the project, NICE developed a structured approach to identify and prioritise which AI use cases for evaluation. The triage tool was co-developed iteratively with NICE colleagues and external stakeholders with expertise in HTA, clinical evaluation, industry and AI implementation. All stakeholder groups emphasised the need for fail-fast evaluation to facilitate agile piloting of AI applications.
RESULTS: The triage tool has four dimensions: impact (e.g., time savings, improved accuracy), readiness (e.g., tool maturity, system willingness), prevalence (e.g., how frequently the activity is undertaken), and risk (e.g., operational, legal). Impact and prevalence are used to determine priority (low, medium, high). Readiness determines when to assess a use case, not whether to assess it. Risk determines how to assess a use case, not whether to assess it. The tool has been used to triage over 90 AI applications at NICE. Prioritised AI applications undergo staged validation, starting with a proof-of-concept process evaluation to establish feasibility, followed by shadow testing to estimate longer-term outcomes.
CONCLUSIONS: NICE’s triage tool provides a structured, pragmatic approach for prioritising AI applications by balancing potential impact and prevalence with readiness and risk. By supporting staged, fail-fast evaluation rather than excluding higher-risk applications upfront, it may help HTA agencies evaluate AI opportunities in a resource-sensitive and proportionate way. This tool will be included in the upcoming NICE AI Methods for Evidence Framework.
METHODS: A NICE HTA Innovation Laboratory (HTA Lab) project, “Future of HTA with AI”, examined the potential role of AI in supporting and evolving health technology assessment (HTA) methods. The project included a landscape review, external stakeholder workshops, and internal consultation. As part of the project, NICE developed a structured approach to identify and prioritise which AI use cases for evaluation. The triage tool was co-developed iteratively with NICE colleagues and external stakeholders with expertise in HTA, clinical evaluation, industry and AI implementation. All stakeholder groups emphasised the need for fail-fast evaluation to facilitate agile piloting of AI applications.
RESULTS: The triage tool has four dimensions: impact (e.g., time savings, improved accuracy), readiness (e.g., tool maturity, system willingness), prevalence (e.g., how frequently the activity is undertaken), and risk (e.g., operational, legal). Impact and prevalence are used to determine priority (low, medium, high). Readiness determines when to assess a use case, not whether to assess it. Risk determines how to assess a use case, not whether to assess it. The tool has been used to triage over 90 AI applications at NICE. Prioritised AI applications undergo staged validation, starting with a proof-of-concept process evaluation to establish feasibility, followed by shadow testing to estimate longer-term outcomes.
CONCLUSIONS: NICE’s triage tool provides a structured, pragmatic approach for prioritising AI applications by balancing potential impact and prevalence with readiness and risk. By supporting staged, fail-fast evaluation rather than excluding higher-risk applications upfront, it may help HTA agencies evaluate AI opportunities in a resource-sensitive and proportionate way. This tool will be included in the upcoming NICE AI Methods for Evidence Framework.
Conference/Value in Health Info
2026-11, ISPOR Europe 2026, Vienna, Austria
Value in Health, Volume 29, Issue 12S
Code
PT7
Topic
Health Technology Assessment, Organizational Practices
Topic Subcategory
Systems & Structure
Disease
No Additional Disease & Conditions/Specialized Treatment Areas