Presentation (CTI)

OBJECTIVES: Statistical Analysis Plans (SAP) for Patient Report Outcome (PRO) analyses are essential for ensuring transparency, replicability, and acceptability of analyses from a regulatory perspective. Developing PRO-specific SAP content is a resource-intensive and time-consuming process, requiring careful specification of estimands, strategies for handling intercurrent events, approaches to missing data, multiplicity adjustments, and sensitivity analyses. This study evaluates the use of multi-agent generative artificial intelligence (GenAI) with Retrieval-Augmented Generation (RAG) to support the development of PRO-related SAP components for clinical and health technology assessment submissions.
METHODS: A structured, prompt-driven GenAI framework was developed to support generation of PRO-related Statistical Analysis Plan content. The framework incorporated predefined prompts aligned with regulatory and methodological guidance to generate key SAP elements, including PRO endpoint definitions, analysis populations, estimands, descriptive and longitudinal analyses, primary and secondary efficacy analyses, responder analyses, subgroup and time-to-event analyses, approaches to missing data, multiplicity control strategies, sensitivity analyses, and graphical outputs. Iterative feedback loops were incorporated to refine generated content across development stages. All AI-generated outputs underwent systematic validation through subject matter expert review and comparison with conventionally developed SAPs to assess methodological alignment, internal consistency, and analytical completeness.
RESULTS: GenAI-assisted development successfully generated PRO-focused SAP content aligned with expert-developed strategies, methods appropriateness, and consistency. The time required to draft and develop was less than that of traditional methods. The alignment was good for more standardized parts but needed expert input for more complex sections such as defining estimand and sensitivity analysis.
CONCLUSIONS: GenAI-based frameworks can speed up the development of PRO-related SAPs while upholding methodological standards. However, some sections that require complex clinical judgment still present limitations and therefore continue to require expert involvement. Future development will focus on supporting more complex analytical scenarios and deeper integration of regulatory guidance within AI-assisted SAP workflows.