OBJECTIVES: For chemotherapy patients, unplanned emergency department (ED) visits and inpatient hospitalization (IP) stays are common and costly. Recent evidence suggests a deep learning algorithm (the Reverse Time Attention (RETAIN) model) can identify patients at elevated risk of adverse events to guide more efficient patient services. This study aims to investigate the intrinsic net benefit of a deep learning algorithm (i.e., RETAIN model) and the economic costs for a given threshold selected by the clinician.

METHODS: Across a range of plausible thresholds, calculate the net benefit of the deep learning algorithm utilizing decision curve analysis. In addition, calculate the economic costs of three clinical strategies: treat according to the algorithm (i.e., Select patients for proactive symptom monitoring according to the RETAIN prediction model), treat all (i.e., Proactive symptom management given to all patients every day), and treat none (i.e., None of the patients receive the proactive symptom management).

RESULTS: For IP stays at a threshold of 5%, the net benefit of the deep learning algorithm (strategy 1) relative to proactive symptom management given to all patients every day (strategy 2) was 0.042 (i.e., 80 fewer false-positive results per 100 patient visit days). IP stay economic cost savings are highest at threshold probabilities between one and five percent. For ED visits, the model shows a modest net benefit between one and two percent.

CONCLUSIONS: This investigation shows the RETAIN prediction model generates a higher net benefit relative to the Treat all and Treat none strategies at low probability thresholds for IP and ED visits and could be used to prevent hospitalization in a clinical setting.