Evaluation of Machine Learning Approaches in Predicting the Initial Treatment Strategy in Patients with Multiple Sclerosis

OBJECTIVES: Although machine learning (ML) models are often used for identifying disease diagnosis and progression, the application of ML in predicting treatment selection remains underexplored. Considering the benefits of early intervention, choosing between the two initial treatment strategies (moderate-efficacy disease-modifying agents [meDMAs] vs. high-efficacy disease-modifying agents [heDMAs]) could be crucial for multiple sclerosis (MS) management. This study evaluated the ML approaches in predicting the initial treatment strategy for MS patients.

METHODS: A retrospective cohort study was conducted using the IBM MarketScan Commercial Claims Database. Adult MS patients who initiated DMA prescriptions between 2016 and 2019 were selected, and the earliest DMA prescription date was assigned as the index date. Three ML models (Random Forests [RF], Extreme Gradient Boosting [XGBoost], and Rule-based model [RBM]) were used to predict whether patients would start on meDMAs or heDMAs. During the 12-month baseline period, 90 factors were collected, including patients’ sociodemographic characteristics, comorbidities, MS-related clinical characteristics, and healthcare utilization. Models were trained on 70% of randomly partitioned data and assessed on the remaining data by Area Under the Curve (AUC), accuracy, and F-1 score.

RESULTS: Out of 10,003 eligible MS patients, 22.92% initiated heDMAs. The model performance measures were comparable in XGBoost (AUC 85%, accuracy 82%, and F1 score 56%), RF model (AUC 84%, accuracy 75%, and F1 score 62%), and RBM model (AUC 84%, accuracy 81%, and F1 score 43%). The number of MS-related outpatient visits, MS-related symptoms, and comorbidities were commonly found to be important factors influencing the selection of initial treatment strategy.

CONCLUSIONS: All three ML approaches, XGBoost, RF, and RBM, had a comparable performance in predicting the initial treatment strategy in MS, emphasizing the feasibility of using ML models in clinical decision-making. Future research should focus on expanding the application of ML in predicting treatment outcomes to optimize individualized care.