Presentation (CTI)

OBJECTIVES: Statistical Analysis Plans (SAPs) play a critical role in ensuring that real-world data (RWD) studies meet the transparency and methodological rigor expected by health technology assessment (HTA) bodies. However, SAP review processes are often unstructured and inconsistent, making it difficult to ensure alignment with HTA-relevant methodological standards. We developed a semi-automated process that supports SAP review before, during, and after manual comment entry, with a specific focus on promoting consistency with HTA guidance and decision-maker expectations.
METHODS: The review process begins with a Word-based macro anchored by a scalable library of ~100 predefined quality assurance review items, organized across 12 methodological themes and mapped to categories relevant to HTA and regulatory guidance documents (e.g., NICE, EUnetHTA, relevant RWE reporting standards). The macro scans SAP headers and uses text-matching logic to auto-insert review items as comment balloons attached to each SAP header. These serve as prompts during manual review, guiding reviewers toward HTA-relevant methodological considerations. Reviewers then add additional comment balloons, each labeled with a “Comment Balloon Header” from the library, optionally adding free-text feedback. A Python-based script extracts all comments and classifies them by action type (“Addressed in SAP” or “Comment to address”) and directive strength (“Flag” or “Suggestion”). Final outputs are provided as a filterable Excel workbook (with a full comment table, Review Item Summary, and Theme Summary), along with a Word-based thematic summary highlighting the most prominent themes to address.
RESULTS: The process has been applied across multiple SAPs supporting RWD-based HTA submissions and related regulatory processes. It reduced review time by 30-40%, improved intra-reviewer consistency, and strengthened alignment between SAP content and HTA-relevant methodological domains.
CONCLUSIONS: By supporting both review preparation and structured feedback synthesis, this hybrid manual-automated SAP review process facilitates prioritization, enhances methodological transparency, and helps ensure that RWD analyses meet HTA body expectations for quality and reproducibility.