ASSESSING ACCURACY, CLINICAL UTILITY, AND FAIRNESS OF RISK STRATIFICATION ALGORITHMS FOR CHRONIC OBSTRUCTIVE PULMONARY DISEASE (COPD) EXACERBATIONS
Author(s)
Crystal Leung, BPSc1, Hana Mullerova, PharmD, PhD2, Christer Janson, MD, PhD3, Pierre-Regis Burgel, MD, PhD4, Alberto Papi, MD, PhD5, Jose Maria Olaguibel, MD, PhD6, Ulla Møller Weinreich, PhD7, Mohsen Sadatsafavi, PhD, MD1.
1The University of British Columbia, Vancouver, BC, Canada, 2AstraZeneca, Cambridge, United Kingdom, 3Uppsala University, Uppsala, Sweden, 4Université Paris Cité, Paris, France, 5University of Ferrara, Ferrara, Italy, 6Hospital Universitario de Navarra, Pamplona, Spain, 7Aalborg University Hospital, Aalborg, Denmark.
1The University of British Columbia, Vancouver, BC, Canada, 2AstraZeneca, Cambridge, United Kingdom, 3Uppsala University, Uppsala, Sweden, 4Université Paris Cité, Paris, France, 5University of Ferrara, Ferrara, Italy, 6Hospital Universitario de Navarra, Pamplona, Spain, 7Aalborg University Hospital, Aalborg, Denmark.
OBJECTIVES: Effective and equitable COPD management is predicated on accurate and fair risk stratification of patient’s future exacerbations. Standard of care uses exacerbation history for stratification, though multivariable models are proposed to improve efficiency. Whether multivariate risk prediction reduces subgroup differences in risk stratification accuracy remains unknown. We evaluated the performance and fairness of exacerbation history and a multivariate risk scoring tool (ACCEPT, Adibi et al, Lancet Respiratory Medicine, 2020) across age, sex, socioeconomic status, and ethnicity among COPD patients in NOVELTY (NOVEL observational longiTudinal studY).
METHODS: We conducted mixed-effects time-to-event analyses including COPD patients aged ≥40 years and sufficient follow-up (≥30 days). The primary outcome was the occurrence of any moderate-to-severe exacerbation within 12 months. We compared exacerbation history based on Global Initiative for Chronic Obstructive Lung Disease (GOLD) 2023 (high-risk: ≥2 moderate/≥1 severe in the past year) and 2026 (≥1 moderate/severe exacerbations in the past year) guidelines, and the global and country-adjusted versions of ACCEPT. For each algorithm, we assessed residual heterogeneity in risk with likelihood ratio test and performance with time-dependent area under the curve (AUC) and net benefit at plausible risk thresholds (6-24%).
RESULTS: We included 5,181 patients (59.4% male; mean age 66.5) from 18 countries (mean follow-up 357 days); 12-month exacerbation risk was 0.258. Substantial residual heterogeneity was observed in sex and race subgroups for both GOLD definitions (p<0.01), and in race for ACCEPT without country-adjustment, but not in ACCEPT after country-adjustment. GOLD2023, GOLD2026 and ACCEPT’s AUC were 0.634 (95%CI 0.619-0.649), 0.681 (0.666-0.697), and 0.731 (0.714-0.747), respectively. In net benefit analysis, country-adjusted ACCEPT had the greatest net benefit across all risk thresholds examined.
CONCLUSIONS: Multivariable prediction models improve risk stratification performance and fairness compared to current exacerbation history classifiers. Country-specific recalibration may be required for improving model performance and calibration without adjusting for more sensitive variables.
METHODS: We conducted mixed-effects time-to-event analyses including COPD patients aged ≥40 years and sufficient follow-up (≥30 days). The primary outcome was the occurrence of any moderate-to-severe exacerbation within 12 months. We compared exacerbation history based on Global Initiative for Chronic Obstructive Lung Disease (GOLD) 2023 (high-risk: ≥2 moderate/≥1 severe in the past year) and 2026 (≥1 moderate/severe exacerbations in the past year) guidelines, and the global and country-adjusted versions of ACCEPT. For each algorithm, we assessed residual heterogeneity in risk with likelihood ratio test and performance with time-dependent area under the curve (AUC) and net benefit at plausible risk thresholds (6-24%).
RESULTS: We included 5,181 patients (59.4% male; mean age 66.5) from 18 countries (mean follow-up 357 days); 12-month exacerbation risk was 0.258. Substantial residual heterogeneity was observed in sex and race subgroups for both GOLD definitions (p<0.01), and in race for ACCEPT without country-adjustment, but not in ACCEPT after country-adjustment. GOLD2023, GOLD2026 and ACCEPT’s AUC were 0.634 (95%CI 0.619-0.649), 0.681 (0.666-0.697), and 0.731 (0.714-0.747), respectively. In net benefit analysis, country-adjusted ACCEPT had the greatest net benefit across all risk thresholds examined.
CONCLUSIONS: Multivariable prediction models improve risk stratification performance and fairness compared to current exacerbation history classifiers. Country-specific recalibration may be required for improving model performance and calibration without adjusting for more sensitive variables.
Conference/Value in Health Info
2026-11, ISPOR Europe 2026, Vienna, Austria
Value in Health, Volume 29, Issue 12S
Code
MSR209
Topic
Clinical Outcomes, Epidemiology & Public Health, Methodological & Statistical Research
Topic Subcategory
Artificial Intelligence, Machine Learning, Predictive Analytics
Disease
Respiratory-Related Disorders (Allergy, Asthma, Smoking, Other Respiratory)