Presentation (CTI)

OBJECTIVES: Health technology assessment (HTA) agencies (NICE, CADTH, etc.), review sponsor-submitted evidence and generate clarification questions to identify evidence gaps and inform reimbursement decisions. This study developed and validated a multi-agent generative AI (GenAI) framework that simulates sponsor-NICE interactions by generating clarification questions from sponsor-submitted data.
METHODS: A multi-agentic GenAI large language models (LLMs) framework was developed to replicate sponsor-HTA interactions. Distinct AI agents reflecting HTA review roles, including the HTA secretariat (lead), clinical, economic, patient-reported outcomes, and patient/public involvement reviewers were included. The HTA secretariat coordinated the process and compiled clarification questions aligning HTA guidance. Using sponsor-submitted evidence for renal cell carcinoma (RCC), the framework generated clarification questions across domains, which were reviewed by subject matter experts (SME) for relevance, accuracy, traceability, data gaps, and alignment with HTA
RESULTS: NICE TA858 for first-line advanced RCC was used to replicate sponsor-HTA interactions, with a multi-agent GenAI framework simulated structured clarification questions using sponsor-submitted evidence. In total, the AI-framework generated 40 questions (12 clinical-effectiveness, 10 cost-effectiveness, 18 textual/additional), compared with 29 questions (12 clinical-effectiveness, 10 cost-effectiveness, 7 textual/additional) raised by NICE. Most AI-generated questions addressed data gaps, with fewer focused on methodological issues. Some NICE textual questions were not generated by the AI-framework, likely due to data masking, reliance on appendix data, or specific data requests. The SME validated the output, showed 80-85% agreement with clarification questions raised by NICE. The additional questions generated by the GenAI demonstrated its capability to identify potential evidence gaps beyond those identified by historical NICE queries and to suggest timely mitigation strategies during reimbursement process.
CONCLUSIONS: The multi-agent GenAI effectively simulates sponsor-HTA interactions, identifying evidence gaps and suggesting mitigation strategies. Beyond replicating NICE interactions, it has the potential to streamline assessments across diverse HTA contexts, with future updates to incorporate budget impact and real-world evidence.