Application of Machine Learning Models to Evaluate COVID-19 Related ICU Utilization in a US Population
Author(s)
Icten Z*1;Chan C1;Munsell M1, Menzin J2
1Boston Health Economics, Boston, MA, USA, 2Boston Health Economics, LLC, Boston, MA, USA
OBJECTIVES: Prediction models that can accurately estimate the risk of poor prognosis could assist providers with proactively managing the at-risk population for the novel coronavirus disease 2019 (COVID-19). This study sought to identify predictors of intensive care unit (ICU) utilization among COVID-19 patients using Optum® de-identified COVID-19 Electronic Health Record (EHR) dataset (2007-2020) and machine learning (ML) techniques.
METHODS: The Optum® COVID-19 EHR dataset from 2019/10/01 through 2020/05/22 was used to identify patients: (1) with an inpatient hospitalization (IP) and positive COVID-19 test, (2) treated in an integrated delivery network, (3) ≥18 years old and (4) with ≥1 encounter (for any reason) during the 12 months (baseline) prior to the start their IP visit (index date). Data were partitioned into train (60%) and validate (40%) sets. Predictors used in ML models included demographics, comorbidities, medications, skilled nursing facility (SNF) admission source, and average BMI. LASSO, XGBoost, support vector machines and random forest (RF) models were built, and the best model was selected using the area under the ROC curve (AUC). Accuracy, average precision (AP), recall, precision, F1-score and specificity were assessed.
RESULTS: The study included 8,292 patients (mean age=62.8 years; females=49%) and 13.9% of the patients had evidence of ICU utilization. The best performing model, RF, had AUC= 73.4%, accuracy=58.5%, AP=29.1%, recall=80.2%, precision=22.7%, F1=0.35 and specificity=54.9%. The RF predictors positively associated with ICU utilization included older age, male gender, SNF admission source, being from Midwest, history of nicotine dependence, chronic obstructive pulmonary disease, use of bronchodilators, diabetes and metabolic disease (long term/current use of insulin, higher BMI), hyperlipidemia, hypertension, sleep apnea, gastro-esophageal reflux and narcotic analgesics utilization.
CONCLUSIONS: Our findings, while generally consistent with known risk factors for COVID-19, such as older age, pulmonary disease and diabetes, also point to other comorbidities, including hyperlipidemia, sleep apnea and reflux, which are worthy of further exploration.
METHODS: The Optum® COVID-19 EHR dataset from 2019/10/01 through 2020/05/22 was used to identify patients: (1) with an inpatient hospitalization (IP) and positive COVID-19 test, (2) treated in an integrated delivery network, (3) ≥18 years old and (4) with ≥1 encounter (for any reason) during the 12 months (baseline) prior to the start their IP visit (index date). Data were partitioned into train (60%) and validate (40%) sets. Predictors used in ML models included demographics, comorbidities, medications, skilled nursing facility (SNF) admission source, and average BMI. LASSO, XGBoost, support vector machines and random forest (RF) models were built, and the best model was selected using the area under the ROC curve (AUC). Accuracy, average precision (AP), recall, precision, F1-score and specificity were assessed.
RESULTS: The study included 8,292 patients (mean age=62.8 years; females=49%) and 13.9% of the patients had evidence of ICU utilization. The best performing model, RF, had AUC= 73.4%, accuracy=58.5%, AP=29.1%, recall=80.2%, precision=22.7%, F1=0.35 and specificity=54.9%. The RF predictors positively associated with ICU utilization included older age, male gender, SNF admission source, being from Midwest, history of nicotine dependence, chronic obstructive pulmonary disease, use of bronchodilators, diabetes and metabolic disease (long term/current use of insulin, higher BMI), hyperlipidemia, hypertension, sleep apnea, gastro-esophageal reflux and narcotic analgesics utilization.
CONCLUSIONS: Our findings, while generally consistent with known risk factors for COVID-19, such as older age, pulmonary disease and diabetes, also point to other comorbidities, including hyperlipidemia, sleep apnea and reflux, which are worthy of further exploration.
Conference/Value in Health Info
2020-11, ISPOR Europe 2020, Milan, Italy
Value in Health, Volume 23, Issue S2 (December 2020)
Code
ML4
Topic
Clinical Outcomes, Epidemiology & Public Health, Health Service Delivery & Process of Care, Methodological & Statistical Research
Topic Subcategory
Artificial Intelligence, Machine Learning, Predictive Analytics, Clinical Outcomes Assessment, Disease Management, Prevalence, Incidence & Disease Risk Factors
Disease
Infectious Disease (non-vaccine), Personalized and Precision Medicine, Respiratory-Related Disorders