Presentation (CTI)

OBJECTIVES: Small changes to wording in marketing authorisation labels and HTA approved therapeutic indications can dramatically restrict which patients can access novel therapies. This study leverages LLMs and ML techniques to (i) create an interpretable pipeline that captures differences in therapeutic indication text between marketing authorisation label and HTA recommendations; ii) identify the most frequent sources of mismatch (e.g., biomarker, line of therapy); iii) facilitate identification of attributes benefiting most from human-in-the-loop feedback.
METHODS: A working dataset of 730 NICE assessments were pulled from the HTA-Hive database. Each corresponding MA and HTA indication were decomposed into seven predefined slots: therapy type, intended use, disease, disease stage, line of therapy, biomarker expression, and population. The extract slot values were populated with rule-based patterns, with off the shelf Large Language Models (LLMs) utilised when rule confidence fell below 0.6. Slot-level similarity metrics (e.g., SapBERT cosine, ordinal distance) were combined with Siamese-BioBERT full indication text similarity and fed into a binary classifier. Cross-validation evaluated binary “match/no-match” performance, while slot-level F1 scores, SHAP values, and template-based rationales clarified the specific sources of agreement or disagreement.
RESULTS: In an initial pilot on 50 manually annotated MA-HTA pairs, 23 HTA recommended indications covered the same patient population as the MA label, while 27 had additional restrictions applied. The hybrid model achieved significantly higher accuracy at identifying differences than a TF-IDF methodology using similarity score cut-offs. Early error analysis suggests biomarker wording and line-of-therapy qualifiers are the most common drivers of mismatch.
CONCLUSIONS: Hybrid ML/LLM approaches, including slot-based mix of rules, biomedical embeddings and LLM prompts can improve analysis and detection of population restrictions in HTA outcomes. Scaling this approach across additional geographic settings could facilitate broader understanding of differences in access to medicines across settings.