GREAT ROC! SO WHAT? BRINGING PREDICTIVE MODELING AND RISK SCORES INTO THE PHYSICIANS' OFFICE
Author(s)
Kohli M*1;Bancroft T2;Riedel A2;Drollinger K2, Wilson J2 1OptumInsight, Burlington, ON, Canada, 2OptumInsight, Eden Prairie, MN, USA
Presentation Documents
OBJECTIVES: To develop and validate a 30-day readmission risk prediction model for implementation in a site-of-service personalized medicine tool. The readmission risk score (RRS) is easily interpretable by providers and adaptable to variability in input data. METHODS: The RRS was developed and validated using administrative claims linked to additional data for commercial and Medicare Advantage (MA) enrollees hospitalized between 01Jan2007 and 30Sep2011. Blocks of candidate predictors (demographics, comorbidities, social determinants of health, mental health conditions, pre-admission service utilization, admission details) were used in logistic regression predicting readmission. Main and interaction effects were examined within blocks and blocks were combined in step-wise fashion. Goodness-of-fit was assessed with Wald chi-square and c-statistics. Sub-models for each combination of blocks accommodated variations in available data. The ROC distribution was used to create an indicator of the utility of each patients’ RRS: green for best predictive power, yellow, and red for predictive power too low for use by the care manager in prioritizing patient outreach. RESULTS: Overall 30-day readmission rate among hospitalized subjects (N=973,316, 53% female, mean age 55, 8% MA, 13% non-white, mean Charlson score: 1.66) was 7.5%. When subjects with a predicted RRS ≥ 0.10 were classified as ‘readmitted,’ model statistics were: c-statistic=0.6735, sensitivity=0.3801, specificity=0.8426. Sub-models were classified as having low, moderate, or best predictive ability (c-statistic ranges: 0.5644–0.6002; 0.6183–0.6483; 0.6583–0.6725 respectively). CONCLUSIONS: The RRS achieved high specificity; sensitivity and overall c-statistic were not optimal. Predictive ability of the index may be improved in the next phase when both claims and electronic medical records data will be used to reconstruct the RRS. The RRS allows care systems to calculate a risk score at the site-of-service with even a minimum set of data. Providers are presented with the strength of the score so that they may apply limited resources with data-driven intelligence.
Conference/Value in Health Info
2013-05, ISPOR 2013, New Orleans, LA, USA
Value in Health, Vol. 16, No. 3 (May 2013)
Code
PHP132
Topic
Health Service Delivery & Process of Care
Topic Subcategory
Quality of Care Measurement
Disease
Multiple Diseases