PACE in Scotland: Utilizing Machine Learning to Assess the Importance of Patient and Clinician Engagement in HTAs in SMC
Author(s)
Emer Gribbon, MSc, BSc1, Leela Barham, MSc Health Economics, BSc Economics2, Brenda Dooley, MSc3.
1Senior Associate, AXIS - The Reimbursement Experts, Dublin 2, Ireland, 2Independent, Royston, United Kingdom, 3AXIS - The Reimbursement Experts, Dublin 2, Ireland.
1Senior Associate, AXIS - The Reimbursement Experts, Dublin 2, Ireland, 2Independent, Royston, United Kingdom, 3AXIS - The Reimbursement Experts, Dublin 2, Ireland.
OBJECTIVES: The Scottish Medicines Consortium (SMC) assesses new medicines for the National Health Service (NHS) in Scotland. A Patient and Clinician Engagement (PACE) meeting can be convened for medicines for end of life and rare conditions allowing patient groups and clinicians further input into decision making. The frequency of PACE meetings was assessed over the last decade and consideration given to PACE outputs in SMC decisions.
METHODS: Detailed advice documents (DADs) from the SMC website (n=742) between 2015 and 2025 were collated in Excel® for descriptive analysis. A subsample of DADs for full submissions (n=250, from 2021 to 2025) were collated for machine learning (ML). DADs were pre-processed in Python®. Descriptive analysis assessed average number of “PACE” mentions per DAD. A text matrix was produced to create models and assess the feature importance of PACE in decisions: Gaussian Naïve Bayes (GNB), Random Forest Classifier (RFC), Logistic Regression (LR).
RESULTS: Overall, PACE meetings were convened for 35.71% of SMC decisions (excluding withdrawn submissions and discontinued medicines) over the last decade, peaking in 2023 (43.40%). The subsample for ML included 110 DADs, where PACE was mentioned an average of 7.2 times per DAD. Accuracy for all algorithms were assessed: GNB: (Accuracy: 0.676; Recall: 0.68), RFC: (Accuracy: 0.705; Recall: 0.71), LR: (Accuracy: 0.705; Recall: 0.71). Utilising the most accurate algorithm (RFC), PACE was the 126th most important feature regarding SMC decision when predicting HTA outcome from SMC (333 total features). Top features included survival data, PAS price and statistically significant.
CONCLUSIONS: Descriptive analyses show that SMC consistently convene and consider PACE insights in HTA decisions. ML analyses signal that while PACE insights may influence HTA outcomes in Scotland, they are not a main driver of HTA outcome. Stronger considerations include survival data and cost. Further analyses should include larger sample sizes and tuning models to improve accuracy.
METHODS: Detailed advice documents (DADs) from the SMC website (n=742) between 2015 and 2025 were collated in Excel® for descriptive analysis. A subsample of DADs for full submissions (n=250, from 2021 to 2025) were collated for machine learning (ML). DADs were pre-processed in Python®. Descriptive analysis assessed average number of “PACE” mentions per DAD. A text matrix was produced to create models and assess the feature importance of PACE in decisions: Gaussian Naïve Bayes (GNB), Random Forest Classifier (RFC), Logistic Regression (LR).
RESULTS: Overall, PACE meetings were convened for 35.71% of SMC decisions (excluding withdrawn submissions and discontinued medicines) over the last decade, peaking in 2023 (43.40%). The subsample for ML included 110 DADs, where PACE was mentioned an average of 7.2 times per DAD. Accuracy for all algorithms were assessed: GNB: (Accuracy: 0.676; Recall: 0.68), RFC: (Accuracy: 0.705; Recall: 0.71), LR: (Accuracy: 0.705; Recall: 0.71). Utilising the most accurate algorithm (RFC), PACE was the 126th most important feature regarding SMC decision when predicting HTA outcome from SMC (333 total features). Top features included survival data, PAS price and statistically significant.
CONCLUSIONS: Descriptive analyses show that SMC consistently convene and consider PACE insights in HTA decisions. ML analyses signal that while PACE insights may influence HTA outcomes in Scotland, they are not a main driver of HTA outcome. Stronger considerations include survival data and cost. Further analyses should include larger sample sizes and tuning models to improve accuracy.
Conference/Value in Health Info
2025-11, ISPOR Europe 2025, Glasgow, Scotland
Value in Health, Volume 28, Issue S2
Code
MSR162
Topic
Health Technology Assessment, Methodological & Statistical Research
Topic Subcategory
Artificial Intelligence, Machine Learning, Predictive Analytics
Disease
No Additional Disease & Conditions/Specialized Treatment Areas