Predictive Algorithms in HTA: What Factors Influence Decision Making in NICE, SMC, and NCPE?
Author(s)
Emer Gribbon, MSc, BSc1, Brenda Dooley, MSc2.
1Senior Associate, AXIS - The Reimbursement Experts, Dublin 2, Ireland, 2AXIS - The Reimbursement Experts, Dublin 2, Ireland.
1Senior Associate, AXIS - The Reimbursement Experts, Dublin 2, Ireland, 2AXIS - The Reimbursement Experts, Dublin 2, Ireland.
OBJECTIVES: Health Technology Assessments (HTA) on new medicines are conducted in the UK by the National Institute for Health and Care Excellence (NICE) and the Scottish Medicines Consortium (SMC) and in Ireland by the National Centre for Pharmacoeconomics (NCPE). Assessors issue reimbursement recommendations and summary reports. This research aims to build algorithms that predict HTA outcome and assess feature importance of phrases within reports across the three jurisdictions.
METHODS: Summary reports between 2021 and 2025 were selected and collated in Excel® from the NICE (n=299), SMC (n=250) and NCPE (n=110) websites. After text preprocessing in Python®, text matrices were produced for exploration. Utilising machine learning, four model predictions were created for each jurisdiction with the text matrices: Gaussian Naïve Bayes (GNB), Random Forest Classifier (RFC), Logistic regression (LR), Linear Support Vector Classifier (SVC).
RESULTS: The accuracy (0-1) of the models for each jurisdiction were: NICE: (GNB: 0.517; RFC: 0.928; LR: 0.574; SVC: 0.617), SMC (GNB: 0.60; RFC: 0.835; LR: 0.612; SVC: 0.624), NCPE: (GNB: 0.506; RFC: 0.716; LR: 0.584; SVC: 0.611). The algorithms presented a class imbalance. Therefore, an oversampling method (Random Oversampling) was implemented to improve the models’ accuracies, which significantly improved the accuracy of each model. Furthermore, the precision and recall for each recommendation across all algorithms improved with oversampling techniques. Utilising the most accurate algorithm (RFC), important phrases affecting recommendation in the three jurisdictions included predicted budget, publish PAS, cost effectiveness improved, managed access, trial designed, case estimated, bias applicant, treatment comparison, and likelihood uncertainty.
CONCLUSIONS: Analysis shows that most HTAs in Ireland require improvement of cost-effectiveness results. Managed Access Protocols (MAPs) also influence recommendations. Similarly, price and cost-effectiveness drive HTA outcomes in the UK, with budget impact and Patient Access Schemes (PAS) influencing outcomes. Further analysis will include tuning parameters and improved oversampling techniques.
METHODS: Summary reports between 2021 and 2025 were selected and collated in Excel® from the NICE (n=299), SMC (n=250) and NCPE (n=110) websites. After text preprocessing in Python®, text matrices were produced for exploration. Utilising machine learning, four model predictions were created for each jurisdiction with the text matrices: Gaussian Naïve Bayes (GNB), Random Forest Classifier (RFC), Logistic regression (LR), Linear Support Vector Classifier (SVC).
RESULTS: The accuracy (0-1) of the models for each jurisdiction were: NICE: (GNB: 0.517; RFC: 0.928; LR: 0.574; SVC: 0.617), SMC (GNB: 0.60; RFC: 0.835; LR: 0.612; SVC: 0.624), NCPE: (GNB: 0.506; RFC: 0.716; LR: 0.584; SVC: 0.611). The algorithms presented a class imbalance. Therefore, an oversampling method (Random Oversampling) was implemented to improve the models’ accuracies, which significantly improved the accuracy of each model. Furthermore, the precision and recall for each recommendation across all algorithms improved with oversampling techniques. Utilising the most accurate algorithm (RFC), important phrases affecting recommendation in the three jurisdictions included predicted budget, publish PAS, cost effectiveness improved, managed access, trial designed, case estimated, bias applicant, treatment comparison, and likelihood uncertainty.
CONCLUSIONS: Analysis shows that most HTAs in Ireland require improvement of cost-effectiveness results. Managed Access Protocols (MAPs) also influence recommendations. Similarly, price and cost-effectiveness drive HTA outcomes in the UK, with budget impact and Patient Access Schemes (PAS) influencing outcomes. Further analysis will include tuning parameters and improved oversampling techniques.
Conference/Value in Health Info
2025-11, ISPOR Europe 2025, Glasgow, Scotland
Value in Health, Volume 28, Issue S2
Code
MSR172
Topic
Health Technology Assessment, Methodological & Statistical Research
Topic Subcategory
Artificial Intelligence, Machine Learning, Predictive Analytics
Disease
No Additional Disease & Conditions/Specialized Treatment Areas