OBJECTIVES: Multiple myeloma (MM) is a chronic disease with considerable patient heterogeneity, and the optimal treatment sequences for patients with MM are unclear. Reinforcement learning (RL) is a branch of machine learning for estimating optimal decision rules over time that can maximize a future outcome, such as improvement in patient health. While RL has been suggested for identifying optimal treatment sequences by learning from real-world data, its application in administrative claims research is limited. This study sought to use RL to identify patient specific optimal treatment sequences that maximize time to refraction or relapse among MM patients using claims data.

METHODS: This study used the AOK PLUS claims database, covering approximately 3.5 million insured persons in Saxony and Thuringia, Germany. Our approach involved a targeted literature review to identify features of MM patients that are clinically useful for tailoring treatment decisions. Treatment sequences were developed by utilizing MM specific therapies such as proteasome inhibitors and immunomodulators. The outcome of interest was the duration between diagnosis to refraction or relapse status, which was defined as the initiation of the third line of therapy. An RL framework with generalized survival random forest as an estimator was used to identify optimal treatment sequences.

RESULTS: A total of 1,844 patients (mean age: 72 years; 53% male) diagnosed with MM were eligible for inclusion into this study. Commonly occurring comorbidities (e.g., hypertension), symptoms (e.g., back pain) and procedures (e.g., MRI) among MM patients were identified. Preliminary empirical validation using simulated data showed that the method performed well at recovering the correct optimal treatment regimen. Analyses of real-world data are underway.

CONCLUSIONS: The established framework may provide a robust and clinically meaningful approach for estimating personalized treatment regimens that can improve patient outcomes and address heterogeneity in MM and similar disease settings using real-world claims data.