ENHANCING CLINICAL TRIAL ELIGIBILITY CRITERIA DESIGN WITH GENERATIVE ARTIFICIAL INTELLIGENCE AND MACHINE LEARNING
Author(s)
Ying Li, PhD1, Brandon Theodorou, PhD2, Jimeng Sun, PhD2.
1Regeneron Pharmaceuticals, Inc, Tarrytown, NY, USA, 2Keiji.AI, Seattle, WA, USA.
1Regeneron Pharmaceuticals, Inc, Tarrytown, NY, USA, 2Keiji.AI, Seattle, WA, USA.
OBJECTIVES: Eligibility criteria (EC) define the target patient population for clinical trials, balancing recruitment, safety, and scientific rigor. Manual EC design and extraction from trial text are labor-intensive and error-prone. We aimed to develop a system leveraging large language models (LLMs) and clustering to accelerate and optimize trial selection and EC design.
METHODS: We built an AI-assisted EC design platform with five steps: (1) Project setup - trial designers enter a brief protocol summary. (2) Trial search/selection - the system retrieves reference trials from ClinicalTrials.gov or proprietary sources, ranks protocols using embedding-based relevance, and enables filtering and selection. (3) EC extraction/clustering - EC are extracted; clinical entities (e.g., age, BMI, disease concepts) are identified using GPT-4.1 mini and mapped to UMLS CUIs. An agentic LLM merges entities and clusters criteria into topics based on ontology similarity, ranking topics by frequency across selected trials. (4) Drafting - GPT-4.1 summarizes topic-specific evidence to generate draft criteria, incorporating human edits. (5) Export - users download drafted EC with provenance to source trials.
RESULTS: Evaluation used a static process without human-in-the-loop feedback. Two components were assessed: (1) trial similarity ranking using precision at top 10 recommendations, and (2) topic identification using recall against known trial protocols. Three publicly available trials (NCT04835519, NCT06787612, NCT07187401) served as gold standards. The top 10 most similar trials were manually reviewed, yielding a precision of 83.3%. Additionally, 28 of 37 eligibility criteria were captured, corresponding to a recall of 75.7%.
CONCLUSIONS: AI-assisted EC design is feasible and can reduce burden on trial design teams while improving consistency. Further evaluation of the drafting process and large-scale validation are needed to ensure robustness. Integration with real-world data represents a promising direction for future development.
METHODS: We built an AI-assisted EC design platform with five steps: (1) Project setup - trial designers enter a brief protocol summary. (2) Trial search/selection - the system retrieves reference trials from ClinicalTrials.gov or proprietary sources, ranks protocols using embedding-based relevance, and enables filtering and selection. (3) EC extraction/clustering - EC are extracted; clinical entities (e.g., age, BMI, disease concepts) are identified using GPT-4.1 mini and mapped to UMLS CUIs. An agentic LLM merges entities and clusters criteria into topics based on ontology similarity, ranking topics by frequency across selected trials. (4) Drafting - GPT-4.1 summarizes topic-specific evidence to generate draft criteria, incorporating human edits. (5) Export - users download drafted EC with provenance to source trials.
RESULTS: Evaluation used a static process without human-in-the-loop feedback. Two components were assessed: (1) trial similarity ranking using precision at top 10 recommendations, and (2) topic identification using recall against known trial protocols. Three publicly available trials (NCT04835519, NCT06787612, NCT07187401) served as gold standards. The top 10 most similar trials were manually reviewed, yielding a precision of 83.3%. Additionally, 28 of 37 eligibility criteria were captured, corresponding to a recall of 75.7%.
CONCLUSIONS: AI-assisted EC design is feasible and can reduce burden on trial design teams while improving consistency. Further evaluation of the drafting process and large-scale validation are needed to ensure robustness. Integration with real-world data represents a promising direction for future development.
Conference/Value in Health Info
2026-05, ISPOR 2026, Philadelphia, PA, USA
Value in Health, Volume 29, Issue S6
Code
MSR149
Topic
Methodological & Statistical Research
Topic Subcategory
Artificial Intelligence, Machine Learning, Predictive Analytics
Disease
No Additional Disease & Conditions/Specialized Treatment Areas