Presentation (CTI)

OBJECTIVES: HTA bodies face rising submission volumes and fixed review timelines, and assessors must understand complex economic models they did not build before they can critique them. We tested whether a model explainer agent, a conversational AI co-worker that interrogates and explains a submitted model in natural language, could ease this comprehension burden and speed clarification-question drafting, without displacing methodological appraisal.
METHODS: We ran a simulation of an HTA body's first-pass review, the agent supporting the familiarisation an assessor performs before appraisal. The test case was a production-grade Excel cost-effectiveness model in rare disease (Markov cohort, 5 states, lifetime horizon, about 150,000 cells, VBA-automated). The agent parsed the workbook into an interrogable representation and indexed the documentation through retrieval-augmented generation (RAG). Against 30 model elements an assessor must understand, and clarification questions set by a senior assessor, one simulated reviewer worked agent-assisted and a comparable reviewer manually, measuring familiarisation time, drafting time, coverage and traceability. The agent explained the model but never validated or modified it.
RESULTS: In this simulation, the agent-assisted reviewer located and explained 29 of 30 model elements (97%) on first pass, including elements in VBA-automated logic. Time to confirmed familiarisation fell from about 8 hours to 1.5 hours (around 80%), and clarification-question drafting time fell by about 70%. Coverage of the senior reference question list rose to 29 of 30. Every explanation traced to a real cell or documented passage, with no fabrications. The single residual error was caught on verification, and appraisal stayed with the assessor.
CONCLUSIONS: The agent gave fast, source-traceable understanding of an unfamiliar submission, helping a reviewer frame stronger clarification questions while keeping the verdict with the assessor. These results come from a single simulated model not yet evaluated within an HTA body; the next step is prospective evaluation with HTA reviewers across submissions.