SOCIODEMOGRAPHIC DISPARITIES IN DIAGNOSTIC DISCORDANCE BETWEEN RADIOLOGISTS AND A COMMERCIAL AI ALGORITHM FOR PULMONARY EMBOLISM (PE) DETECTION
Author(s)
Greer Williams, BSc1, Amir H. Gandomi, PhD1, Maria X. Sanmartin, PhD1, Shlomit Goldberg-Stein, MD1, Matthew Barish, MD1, Elizabeth Y. Rula, PhD2, Jason J. Naidich, MD1, Pina C. Sanelli, MD1;
1Northwell, New Hyde Park, NY, USA, 2Harvey L. Neiman Health Policy Institute, Reston, VA, USA
1Northwell, New Hyde Park, NY, USA, 2Harvey L. Neiman Health Policy Institute, Reston, VA, USA
OBJECTIVES: As artificial intelligence (AI) tools see increasing adoption in medical imaging, including for pulmonary embolism (PE) detection, understanding potential algorithmic bias is critical for ensuring health equity. This study sought to assess sociodemographic differences in diagnostic discordance between interpreting radiologists and a commercially deployed AI algorithm following its integration within a large U.S. health system.
METHODS: This retrospective observational cohort study included 29,492 unique adult CT pulmonary angiography (CTPA) exams processed between June 2021 and February 2023. The primary outcome was AI-radiologist discordance, defined as any binary mismatch in PE classification (positive vs. negative) between the AI tool output and the final radiologist report. Multivariable logistic regression identified independent predictors of discordance, adjusting for sex, age, race, ethnicity, clinical setting, and time since AI deployment.
RESULTS: The overall AI-radiologist discordance rate was 2.1%. Multivariable analysis identified several significant predictors of higher discordance, including age >65 years (OR=1.65, 95% CI [1.35-2.00], p<0.0001) and Black or African American race (OR=1.25, 95% CI [1.04-1.52], p=0.038). Notably, the odds of discordance decreased significantly over the study period (OR=0.92 per quarter, 95% CI [0.87-0.97], p=0.004).
CONCLUSIONS: The study’s findings highlight significant sociodemographic disparities in the real-world performance of a clinically deployed AI algorithm for PE detection. These results underscore the necessity for continuous monitoring of medical imaging models to mitigate algorithmic bias and support the role of AI as an augmentative tool rather than an autonomous replacement for radiologists.
METHODS: This retrospective observational cohort study included 29,492 unique adult CT pulmonary angiography (CTPA) exams processed between June 2021 and February 2023. The primary outcome was AI-radiologist discordance, defined as any binary mismatch in PE classification (positive vs. negative) between the AI tool output and the final radiologist report. Multivariable logistic regression identified independent predictors of discordance, adjusting for sex, age, race, ethnicity, clinical setting, and time since AI deployment.
RESULTS: The overall AI-radiologist discordance rate was 2.1%. Multivariable analysis identified several significant predictors of higher discordance, including age >65 years (OR=1.65, 95% CI [1.35-2.00], p<0.0001) and Black or African American race (OR=1.25, 95% CI [1.04-1.52], p=0.038). Notably, the odds of discordance decreased significantly over the study period (OR=0.92 per quarter, 95% CI [0.87-0.97], p=0.004).
CONCLUSIONS: The study’s findings highlight significant sociodemographic disparities in the real-world performance of a clinically deployed AI algorithm for PE detection. These results underscore the necessity for continuous monitoring of medical imaging models to mitigate algorithmic bias and support the role of AI as an augmentative tool rather than an autonomous replacement for radiologists.
Conference/Value in Health Info
2026-05, ISPOR 2026, Philadelphia, PA, USA
Value in Health, Volume 29, Issue S6
Code
MSR200
Topic
Methodological & Statistical Research
Topic Subcategory
Artificial Intelligence, Machine Learning, Predictive Analytics
Disease
No Additional Disease & Conditions/Specialized Treatment Areas, SDC: Cardiovascular Disorders (including MI, Stroke, Circulatory), SDC: Respiratory-Related Disorders (Allergy, Asthma, Smoking, Other Respiratory)