MACHINE LEARNING PREDICTION OF NON-INVASIVE VENTILATION FAILURE IN COPD: A TRIPOD-ALIGNED REAL-WORLD ANALYSIS
Author(s)
Komaleshwari Rani, MSc1, Tuli De, PhD2, Jackie Vanderpuye-Orgle, MSc, PhD3.
1Sr. Consultant, Advanced Analytics: RWE Analytics, Parexel International, Bolton, United Kingdom, 2Parexel International, Cupertino, CA, USA, 3PAREXEL, La Verne, CA, USA.
1Sr. Consultant, Advanced Analytics: RWE Analytics, Parexel International, Bolton, United Kingdom, 2Parexel International, Cupertino, CA, USA, 3PAREXEL, La Verne, CA, USA.
OBJECTIVES: Non-invasive ventilation (NIV) fails in 15-40% of COPD (Chronic Obstructive Pulmonary Disease) patients in intensive care, with failure associated with mortality of 25-30% versus 5-10% for NIV success and 5-7 additional ICU days. Existing risk scores, including HACOR, assess risk only at NIV initiation. This study developed and evaluated a machine learning model to predict NIV failure from temporal ICU data, following TRIPOD guidelines.
METHODS: A retrospective cohort of 1,107 COPD ICU admissions (ICD-10 J441) was extracted from MIMIC-IV. NIV failure was defined from intubation and invasive ventilation procedure codes. Predictors (vital signs, FiO2, oxygen delivery device, GCS sub-scores) were engineered as minimum, mean and maximum across four six-hour windows within the first 24 hours of admission, ensuring temporal precedence over the outcome. Leakage-prone and bias-prone variables were excluded. A stratified 70/15/15 train/validation/test split was applied; SMOTE was applied exclusively to the training set (20.6% failure prevalence). Logistic Regression and Random Forest classifiers were trained with grid-search optimisation. The decision threshold (0.60) was optimised on the validation set against pre-specified TRIPOD targets (AUC ≥0.80, sensitivity ≥0.75, specificity ≥0.70) and around 5% optimisation bias is acknowledged. Robustness was confirmed via five-fold cross-validation with SMOTE inside each fold.
RESULTS: Random Forest outperformed Logistic Regression and is reported here. On 167 held-out set (35 failures, 132 successes), the model achieved AUC 0.803 (95% CI 0.724-0.879), sensitivity 0.817 (95% CI 0.744-0.880), and specificity 0.632 (95% CI 0.475-0.788), reflecting a sensitivity-specificity trade-off favouring early detection of high-mortality outcomes. Cross-validation confirmed stability (AUC 0.775±0.009, F1 0.852±0.013). Dominant predictors were oxygen delivery device (W3: 0.068) and GCS verbal response (W4: 0.049), consistent with HACOR components.
CONCLUSIONS: Random Forest model achieved modest-to-good discrimination for NIV failure in COPD. Single-centre US data limits generalisability, and the high false-positive rate should be considered when implementing this model as a decision-support tool.
METHODS: A retrospective cohort of 1,107 COPD ICU admissions (ICD-10 J441) was extracted from MIMIC-IV. NIV failure was defined from intubation and invasive ventilation procedure codes. Predictors (vital signs, FiO2, oxygen delivery device, GCS sub-scores) were engineered as minimum, mean and maximum across four six-hour windows within the first 24 hours of admission, ensuring temporal precedence over the outcome. Leakage-prone and bias-prone variables were excluded. A stratified 70/15/15 train/validation/test split was applied; SMOTE was applied exclusively to the training set (20.6% failure prevalence). Logistic Regression and Random Forest classifiers were trained with grid-search optimisation. The decision threshold (0.60) was optimised on the validation set against pre-specified TRIPOD targets (AUC ≥0.80, sensitivity ≥0.75, specificity ≥0.70) and around 5% optimisation bias is acknowledged. Robustness was confirmed via five-fold cross-validation with SMOTE inside each fold.
RESULTS: Random Forest outperformed Logistic Regression and is reported here. On 167 held-out set (35 failures, 132 successes), the model achieved AUC 0.803 (95% CI 0.724-0.879), sensitivity 0.817 (95% CI 0.744-0.880), and specificity 0.632 (95% CI 0.475-0.788), reflecting a sensitivity-specificity trade-off favouring early detection of high-mortality outcomes. Cross-validation confirmed stability (AUC 0.775±0.009, F1 0.852±0.013). Dominant predictors were oxygen delivery device (W3: 0.068) and GCS verbal response (W4: 0.049), consistent with HACOR components.
CONCLUSIONS: Random Forest model achieved modest-to-good discrimination for NIV failure in COPD. Single-centre US data limits generalisability, and the high false-positive rate should be considered when implementing this model as a decision-support tool.
Conference/Value in Health Info
2026-11, ISPOR Europe 2026, Vienna, Austria
Value in Health, Volume 29, Issue 12S
Code
MSR270
Topic
Methodological & Statistical Research, Real World Data & Information Systems
Topic Subcategory
Artificial Intelligence, Machine Learning, Predictive Analytics, Confounding, Selection Bias Correction, Causal Inference
Disease
Respiratory-Related Disorders (Allergy, Asthma, Smoking, Other Respiratory)