SUPPORTING DECISION-GRADE CLINICAL BENCHMARKING FOR PERCUTANEOUS CORONARY INTERVENTION: RE-DEVELOPMENT AND TEMPORAL VALIDATION OF A NATIONAL RISK MODEL FOR POST-PCI MORTALITY
Author(s)
Ashley K. Clift, MBBS DPhil1, Jack B. Robinson, MD2, William Pratt, PhD3, Mamas A. Mamas, MBChB DPhil4.
1Granta Consultants; Jazz Advisory; Imperial College London, London, United Kingdom, 2Granta Consultants, Cambridge, United Kingdom, 3National Institute for Cardiovascular Outcomes Research, Leicester, United Kingdom, 4National Institute for Cardiovascular Outcomes Research; Keele University, Keele, United Kingdom.
1Granta Consultants; Jazz Advisory; Imperial College London, London, United Kingdom, 2Granta Consultants, Cambridge, United Kingdom, 3National Institute for Cardiovascular Outcomes Research, Leicester, United Kingdom, 4National Institute for Cardiovascular Outcomes Research; Keele University, Keele, United Kingdom.
OBJECTIVES: National clinical audits require precise risk adjustment to ensure equitable benchmarking across centres. The UK National Audit of Percutaneous Coronary Interventions (NAPCI) relies on a 30-day post-PCI mortality model developed over a decade ago. Evolving clinical practice risks calibration drift, potentially misestimating risk in contemporary cohorts. We evaluated the original model's performance against a redeveloped model incorporating novel predictors to support national quality improvement.
METHODS: Data from 918,965 PCI procedures within NAPCI (FY 2014/15 to FY 2024/25) were used for temporal validation. The original logistic regression model was recalibrated using FY 2023/24 data. Data from FY 2014/15 to FY 2023/24 were utilised to redevelop an updated model incorporating contemporary variables (including left ventricular support devices [ECMO, Impella], left main PCI, triple vessel disease, peripheral vascular disease, fractional polynomials for continuous predictors, and updated renal function measures). Both models underwent comparative, temporal validation using held-out FY 2024/25 data. Performance heterogeneity was evaluated across age, sex, deprivation, and ethnicity groupings. Clinical utility was examined using Decision Curve Analysis (DCA).
RESULTS: The original model retained discrimination (AUC: 0.825 [95% CI: 0.822 to 0.828]) but exhibited significant calibration drift in contemporary data (slope: 0.925 [95% CI: 0.911 to 0.936]; intercept: -0.245 [95% CI: -0.263 to -0.227]). The redeveloped model demonstrated slightly higher discrimination (AUC: 0.837 vs. 0.821 for the recalibrated model) and excellent calibration (slope 0.991 [95% CI: 0.950 to 1.030]; intercept 0.066 [95% CI: -0.008 to 0.120]). DCA demonstrated superior net benefit for the redeveloped model across all decision thresholds examined (up to 30%). No predictive bias or performance heterogeneity was observed across examined sub-groups.
CONCLUSIONS: Model calibration drift undermines provider benchmarking and predictor-risk associations evolve over time. Incorporating modern predictors and validating across patient sub-groups restores decision-grade calibration, supporting robust, equitable quality improvement and outcomes assessment at a national scale.
METHODS: Data from 918,965 PCI procedures within NAPCI (FY 2014/15 to FY 2024/25) were used for temporal validation. The original logistic regression model was recalibrated using FY 2023/24 data. Data from FY 2014/15 to FY 2023/24 were utilised to redevelop an updated model incorporating contemporary variables (including left ventricular support devices [ECMO, Impella], left main PCI, triple vessel disease, peripheral vascular disease, fractional polynomials for continuous predictors, and updated renal function measures). Both models underwent comparative, temporal validation using held-out FY 2024/25 data. Performance heterogeneity was evaluated across age, sex, deprivation, and ethnicity groupings. Clinical utility was examined using Decision Curve Analysis (DCA).
RESULTS: The original model retained discrimination (AUC: 0.825 [95% CI: 0.822 to 0.828]) but exhibited significant calibration drift in contemporary data (slope: 0.925 [95% CI: 0.911 to 0.936]; intercept: -0.245 [95% CI: -0.263 to -0.227]). The redeveloped model demonstrated slightly higher discrimination (AUC: 0.837 vs. 0.821 for the recalibrated model) and excellent calibration (slope 0.991 [95% CI: 0.950 to 1.030]; intercept 0.066 [95% CI: -0.008 to 0.120]). DCA demonstrated superior net benefit for the redeveloped model across all decision thresholds examined (up to 30%). No predictive bias or performance heterogeneity was observed across examined sub-groups.
CONCLUSIONS: Model calibration drift undermines provider benchmarking and predictor-risk associations evolve over time. Incorporating modern predictors and validating across patient sub-groups restores decision-grade calibration, supporting robust, equitable quality improvement and outcomes assessment at a national scale.
Conference/Value in Health Info
2026-11, ISPOR Europe 2026, Vienna, Austria
Value in Health, Volume 29, Issue 12S
Code
HSD54
Topic
Health Service Delivery & Process of Care, Methodological & Statistical Research, Study Approaches
Disease
Cardiovascular Disorders (including MI, Stroke, Circulatory)