Session (CTI)

ISSUE: Health economic models can now be rapidly developed using artificial intelligence (AI) agents to create complex cost-effectiveness models (CEMs). This approach can substantially reduce development times and increase model flexibility and scope (as they can be rapidly restructured or rescoped at short notice). However, this approach may lack clinical validity and an inherent understanding of the decision problem, unmet need, and value drivers. LLM-based agents used for model development are also probabilistic and therefore not necessarily reproducible, and the decision-making process for model construction is opaque (with justification of model assumptions being made by humans post-hoc). Large language models (LLMs) on which the AI agents rely may also be biased, based on their training data, and lacking original targeted solutions. These issues need to be resolved before AI-developed CEMs are able to significantly contribute to decision-making within HTAs. This session will detail the current risks and concerns around AI-assisted modeling, and how current methods may be misaligned with the use of AI agents and LLMs. It will also describe what actions need to be taken by relevant stakeholders and what guidelines and guardrails are required to increase confidence in models built using AI and enable them to be used within HTAs. OVERVIEW: Ray Gani will moderate and provide an assessment of current AI capabilities for developing CEMs, and current guidelines and advice on their use within HTA, covering perceived opportunities, ongoing initiatives, and major reservations. Páll Jónsson will describe the steps being taken at NICE to address the current issues described above and potential guidance updates. Ronan Mahon will discuss the implications of different types of risks, in particular fundamental (inherent) vs. manageable (via human oversight). Peter Quon will describe work ongoing within industry to resolve these issues and integrate AI workflows into CEM development. Each speaker will have 10 mins, with 20 mins for audience questions, feedback, and discussion. The intended audience is health economists, health-economic modelers, and HTA organizations.