Development and Validation of a Machine Learning Model for Predicting French HAS ASMR Ratings Using Product and Disease Characteristics
Author(s)
Mondher Toumi, MD, MSc, PhD1, Przemyslaw Lipka, .2, Imen Reguei, PharmD3, Hichem Dridi, Eng3, Samuel Aballea, MSc, PhD2.
1Laboratoire de santé publique, Aix-Marseille University, Marseille, France, 2InovIntell, Kraków, Poland, 3InovIntell, Tunis, Tunisia.
1Laboratoire de santé publique, Aix-Marseille University, Marseille, France, 2InovIntell, Kraków, Poland, 3InovIntell, Tunis, Tunisia.
OBJECTIVES: The French Haute Autorité de Santé (HAS) assigns Amélioration du Service Médical Rendu (ASMR) ratings that determine premium pricing eligibility. ASMR outcomes are difficult to predict, creating uncertainty for health technology developers (HTDs) during product development and investment decisions. This study aimed to develop a machine learning model for predicting ASMR ratings using product and disease characteristics available during early development.
METHODS: Data were extracted from the NaviHTA database, containing structured information from European HTA agency appraisals since 2018. The analysis included 680 HAS appraisals with corresponding ASMR ratings, categorized into three groups: ASMR I-III (major to moderate improvement), ASMR IV (minor improvement), and ASMR V (no improvement). A tree-based gradient boosting model (CatBoost) was developed in Python using iterative cross-validation, where 90% of data trained the model and 10% served for testing across multiple iterations. Initially, 114 predictive features were considered, including standardized treatment effect size measures. Feature selection and importance were assessed using Shapley values to identify the most influential variables for ASMR prediction.
RESULTS: The final model incorporated 34 selected features and achieved an average F1 macro score of 0.89. Prediction accuracy was 84% for products with predicted ASMR I-III, 86% for predicted ASMR IV, and 93% for predicted ASMR V. The features demonstrating highest global Shapley values were general effect size, life-threatening nature of the disease, effect size against mortality, target population size, and unmet medical needs.
CONCLUSIONS: The model can reliably distinguish between different ASMR categories using information typically available in target product profiles during early development stages. The model could enhance HTDs' strategic decision-making capabilities, enabling more efficient resource allocation toward development programs with favorable market access prospects.
METHODS: Data were extracted from the NaviHTA database, containing structured information from European HTA agency appraisals since 2018. The analysis included 680 HAS appraisals with corresponding ASMR ratings, categorized into three groups: ASMR I-III (major to moderate improvement), ASMR IV (minor improvement), and ASMR V (no improvement). A tree-based gradient boosting model (CatBoost) was developed in Python using iterative cross-validation, where 90% of data trained the model and 10% served for testing across multiple iterations. Initially, 114 predictive features were considered, including standardized treatment effect size measures. Feature selection and importance were assessed using Shapley values to identify the most influential variables for ASMR prediction.
RESULTS: The final model incorporated 34 selected features and achieved an average F1 macro score of 0.89. Prediction accuracy was 84% for products with predicted ASMR I-III, 86% for predicted ASMR IV, and 93% for predicted ASMR V. The features demonstrating highest global Shapley values were general effect size, life-threatening nature of the disease, effect size against mortality, target population size, and unmet medical needs.
CONCLUSIONS: The model can reliably distinguish between different ASMR categories using information typically available in target product profiles during early development stages. The model could enhance HTDs' strategic decision-making capabilities, enabling more efficient resource allocation toward development programs with favorable market access prospects.
Conference/Value in Health Info
2025-11, ISPOR Europe 2025, Glasgow, Scotland
Value in Health, Volume 28, Issue S2
Code
HTA107
Topic
Health Technology Assessment, Methodological & Statistical Research
Topic Subcategory
Decision & Deliberative Processes
Disease
No Additional Disease & Conditions/Specialized Treatment Areas