Presentation (CTI)

OBJECTIVES: This study evaluated capabilities and limitations of autonomous AI agents in health economic evaluations identifying areas where human expertise remained indispensable. A multi-agent AI pipeline was built to conduct a cost-effectiveness analysis comparing atomoxetine versus immediate-release methylphenidate in adults with moderate-to-severe ADHD. To our knowledge, this is the first evaluation of an integrated multi-agent large language model (LLM) pipeline spanning evidence retrieval, model development, and automated quality control within a single HTA modelling workflow.
METHODS: Three sequentially operating agentic LLM instances were built (Claude Sonnet 4.6, Anthropic), each assuming a distinct role:Agent 1 (Evidence): Retrieved and reviewed published ADHD literature spanning RCTs, meta-analyses, HTA submissions and real-world UK studies.Agent 2 (Build): Implemented an excel-based structurally complete, NHS perspective cost-effectiveness model using Agent 1’s evidence.Agent 3 (QC): Conducted quality control on Agent 2’s model, reviewing structure and calculations, documenting issues and verifying resolution after amendment.Human expertise was required to validate each agent’s output.
RESULTS: Agent 1 screened 25 ADHD-related UK papers, extracting clinical evidence, model structures and model inputs. Agent 2 generated a NICE-submission-structured cost-effectiveness model implementing a three-state Markov design (responder, non-responder, discontinued), 3.5% discounting, state-specific costs and sensitivity analysis. Agent 3 identified and resolved twelve model issues including half-cycle correction misapplication and transition matrix row-sum violations. Human oversight identified a few invalid references and input extraction errors from Agent 1 while Agent 2-generated VBA code required manual integration into the Excel workbook.
CONCLUSIONS: A structured multi-agent AI pipeline completed most technical tasks required to develop a cost-effectiveness model, including evidence retrieval, model implementation, VBA generation and quality control without step-by-step human instruction. These findings suggest agentic AI could accelerate HTA model development, although human contributions remained essential for structural judgement, parameter validation and VBA integration.