DEVELOPING A REPRODUCIBLE TRANSPORTABILITY FRAMEWORK FOR REAL-WORLD EXTERNAL CONTROL ARMS: AN ADULT SEPSIS EXAMPLE USING MIMIC IV
Author(s)
Nikhil Tiwari, Bachelor's, Shivesh Gupta, Bachelor's.
Frekil, San Francisco, CA, USA.
Frekil, San Francisco, CA, USA.
OBJECTIVES: External control arms (ECAs) increasingly support single-arm trials and HTA, including the EU Joint Clinical Assessment, but their validity requires demonstrating, not assuming transportability through a reproducible pipeline. We built and stress-tested such a workflow, using adult ICU sepsis as a high-dimensional, acute-onset test case.
METHODS: Retrospective cohort, MIMIC-IV v3.1. Adults meeting Sepsis-3 criteria were identified using suspected infection and acute organ dysfunction, with sepsis onset defined as time zero to minimize immortal-time bias. The primary estimand was 28-day all-cause mortality - was estimated by Kaplan-Meier with Greenwood 95% CIs; subgroups were compared by log-rank tests. Missingness >5% (lactate, 37.8%) used multiple imputation by chained equations; others complete-case.
Transportability diagnostics comprised covariate profiling, severity distributions (SOFA, SAPS II, APS III, OASIS), cross-system severity-decile mortality, and a logistic predicted-mortality model (discrimination, C-statistic; calibration, Brier) parameterizing inverse-odds-of-participation weighting, with pre-specified flags (standardized mean difference >0.1; participation C>0.80).
RESULTS: Among 31,910 patients (median age 68.2; median SOFA 3.0; mechanical ventilation 52.1%; vasopressors 41.0%), 28-day mortality was 20.7% (95% CI 20.2-21.1%; 6,597 deaths). With zero pre-28-day censoring, the Kaplan-Meier estimate equaled the crude proportion, removing informative censoring as a validity threat. Mortality rose monotonically with severity (18.5% [SOFA 2-5] to 64.9% [≥15]), and was higher in septic shock (28.0% vs 19.1%) and age ≥65 (24.5% vs 15.4%; log-rank p<0.001). The predicted-mortality model achieved a C-statistic of 0.762. Estimates were robust across sensitivity analyses (medical-ICU-only 27.1%; in-hospital 17.4%).
CONCLUSIONS: This open-data, open-source workflow constructs a benchmark-consistent sepsis ECA and quantifies rather than assumes its transportability via covariate profiling, severity-stratified mortality, and a calibrated predicted-mortality model (C=0.762); inverse-odds-of-participation weighting and covariate-overlap diagnostics (SMD>0.1; participation C>0.80) are pre-specified to execute once a reference trial is named. With rigorous time-zero handling and reproducible code, it offers sponsors and HTA bodies an auditable, health-system-agnostic template for evaluating real-world external controls.
METHODS: Retrospective cohort, MIMIC-IV v3.1. Adults meeting Sepsis-3 criteria were identified using suspected infection and acute organ dysfunction, with sepsis onset defined as time zero to minimize immortal-time bias. The primary estimand was 28-day all-cause mortality - was estimated by Kaplan-Meier with Greenwood 95% CIs; subgroups were compared by log-rank tests. Missingness >5% (lactate, 37.8%) used multiple imputation by chained equations; others complete-case.
Transportability diagnostics comprised covariate profiling, severity distributions (SOFA, SAPS II, APS III, OASIS), cross-system severity-decile mortality, and a logistic predicted-mortality model (discrimination, C-statistic; calibration, Brier) parameterizing inverse-odds-of-participation weighting, with pre-specified flags (standardized mean difference >0.1; participation C>0.80).
RESULTS: Among 31,910 patients (median age 68.2; median SOFA 3.0; mechanical ventilation 52.1%; vasopressors 41.0%), 28-day mortality was 20.7% (95% CI 20.2-21.1%; 6,597 deaths). With zero pre-28-day censoring, the Kaplan-Meier estimate equaled the crude proportion, removing informative censoring as a validity threat. Mortality rose monotonically with severity (18.5% [SOFA 2-5] to 64.9% [≥15]), and was higher in septic shock (28.0% vs 19.1%) and age ≥65 (24.5% vs 15.4%; log-rank p<0.001). The predicted-mortality model achieved a C-statistic of 0.762. Estimates were robust across sensitivity analyses (medical-ICU-only 27.1%; in-hospital 17.4%).
CONCLUSIONS: This open-data, open-source workflow constructs a benchmark-consistent sepsis ECA and quantifies rather than assumes its transportability via covariate profiling, severity-stratified mortality, and a calibrated predicted-mortality model (C=0.762); inverse-odds-of-participation weighting and covariate-overlap diagnostics (SMD>0.1; participation C>0.80) are pre-specified to execute once a reference trial is named. With rigorous time-zero handling and reproducible code, it offers sponsors and HTA bodies an auditable, health-system-agnostic template for evaluating real-world external controls.
Conference/Value in Health Info
2026-11, ISPOR Europe 2026, Vienna, Austria
Value in Health, Volume 29, Issue 12S
Code
MSR11
Topic
Methodological & Statistical Research, Real World Data & Information Systems, Study Approaches
Topic Subcategory
Artificial Intelligence, Machine Learning, Predictive Analytics, Confounding, Selection Bias Correction, Causal Inference
Disease
Geriatrics, Infectious Disease (non-vaccine)