Presentation (CTI)

OBJECTIVES: Health technology assessments (HTA) agencies like National Institute for Health and Care Excellence (NICE) evaluate sponsor-submitted evidence to inform reimbursement decisions, with a key focus on identifying evidence gaps through clarification questions. This study aimed to develop and validate a multi-agent Generative artificial intelligence (GenAI) framework that simulate sponsor HTA conversations by generating clarification questions using sponsor-submitted data.
METHODS: A role-based, multi-agent large language models (LLMs) framework was developed to simulate sponsor-HTA interactions, featuring specialized AI agents representing HTA secretariat (lead), clinical, economic, PRO and patient/public involvement reviewers. The HTA secretariat coordinated the process and compiled clarification questions based on HTA guidelines. Using sponsor-submitted data for hepatocellular carcinoma (HCC), the tool generated questions across clinical, economic, and textual domains. Subject matter experts (SME) validated the outputs for relevance, accuracy, traceability, and alignment with HTA expectations.
RESULTS: Two NICE HTA submissions in first-line advanced/metastatic HCC were selected for sponsor-HTA simulation. The multi-agent GenAI system successfully simulated interactions by generating structured clarification questions based on sponsor-submitted evidence. A total of 104 questions were generated including 29 clinical, 62 economic, and 13 textual. The majority of questions focused on data-related gaps, while a smaller proportion sought clarification on methodological aspects. The AI-generated output was validated by a SME, demonstrating 80-85% agreement with actual clarification questions previously raised by NICE in the technology appraisals. The remaining 15-20% of questions, although not identical to historical NICE examples, were still relevant and evidence-based, reflecting potential gaps that would warrant further clarification within the HTA process.
CONCLUSIONS: The multi-agent GenAI framework provides a scalable, regulatory-aligned method for simulating NICE HTA reviews by generating structured clarification questions. It enhances sponsor preparedness and may streamline assessments, with future updates to include budget impact and real-world evidence.