Development and Validation of a Tool to Predict Short Term Opioid Overdose Risk after Acquiring a Prescribed Controlled Substance

OBJECTIVES: To develop an opioid overdose (OOD) risk prediction tool that is specifically aimed at predicting OOD risk in the near term (7 and 30 days) after acquiring prescribed controlled substances (CS).

METHODS: Arkansas statewide data between 2014 and 2020 was utilized. CS acquisition was assessed using Prescription Drug Monitoring Program (PDMP) data. Linked death certificates, inpatient discharge, and emergency department data were used to identify fatal and non-fatal OOD. Features engineered on each CS dispensing date included patient demographics, characteristics of opioids dispensed, characteristics of other controlled substances dispensed, number of unique prescribers/pharmacies, and measures of prior OOD. The data were randomly split 1:1 into training and test sets. To address severe data imbalance, a 1:100 random under-sampling (RUS) of the majority class was performed in the training set. To optimize model performance, hyperparameter tuning was utilized for training random forest (RF), logistic regression (LR), naïve bayes (NB), and gradient boosting (GB) models. Model discrimination was evaluated on the complete test set using c-statistic, recall, specificity, positive predictive value (PPV), and negative predictive value (NPV).

RESULTS: 2,818,135 persons filled at least one CS during the study period and 8,436 experienced one or more OOD. With a c-statistic of 0.773, the random forest model had the highest discrimination in predicting OOD in the 7 days following CS acquisition (sensitivity=62.44%, specificity=77.40%, PPV=0.02%, NPV=100%). Models based on Gradient Boosting (c-statistic=0.769), Logistic regression (c-statistic=0.767), and naive bayes (c-statistic=0.735) all had lower levels of discrimination.

CONCLUSIONS: It is possible to predict fatal and non-fatal OOD in the periods immediately following the acquisition of controlled substances with reasonably good model discrimination using a limited range of features derived almost exclusively from PDMP records. This tool could be used by health practitioners to identify candidates for naloxone after prescribing or dispensing a controlled substance.