OBJECTIVES: As drug developers or evaluators consider new medical product submissions, e.g., for clinical regulatory agencies or HTAs, an effective process for all stakeholders requires understanding of regulatory precedents for study designs, data sources, outcome measurements, analytical methodology, interpretations of findings, and other diverse and interrelated topics. Human expertise is central to this process, but subject to time and resource constraints, especially when relevant precedents (1) span multiple therapeutic areas and decision authorities, (2) exist only as short passages within large documents, or (3) involve topics poorly-suited to keyword searches. We developed and assessed AI-based approaches to identifying relevant regulatory precedents.

METHODS: Public records were compiled from the US FDA (guidance, briefing documents, drug labels), the EMA (guidance, EPARs, SmPCs), as well as from selected HTAs. To enable rapid search and machine learning applications, text extracted from these documents was organized into an inverted index database with embeddings derived from a medical-specific pre-trained large language model. This platform was used to address questions arising in real-world applications from the drug developer perspective. Use cases were assessed and classified.

RESULTS: The database included over 30 gigabytes of textual data. Common use cases included: (1) identifying documents and passages of text relevant to specific topics via one-shot learning and/or keyword search, (2) exhaustive search for specific concepts (e.g., “use of real-world data”, “performance outcomes”, “survival extrapolation”) in the context of broad therapeutic areas (e.g., “early-stage oncology”, “pediatric diseases”, “adjuvant therapy”) using iterative machine learning from human feedback and (3) generative applications summarizing expected limitations or critiques of data sources and methodologies. Representative examples will be presented to illustrate AI’s strengths and limitations.

CONCLUSIONS: Artificial intelligence can augment human expertise to simultaneously increase the efficiency, breadth and depth of understanding of regulatory precedents in drug development applications.