COMPARING THE PERFORMANCE OF ESTIMATORS FOR TRANSPORTING TRIAL TREATMENT EFFECTS TO TARGET POPULATIONS UNDER VARYING POSITIVITY: A SIMULATION STUDY
Author(s)
Ting-An Tai, MSc1, Yi-Shu Lin, PhD2.
1Astellas Pharma Co. Ltd, Dublin, Ireland, 2School of Health Care Administration, Taipei Medical University, Taipei, Taiwan.
1Astellas Pharma Co. Ltd, Dublin, Ireland, 2School of Health Care Administration, Taipei Medical University, Taipei, Taiwan.
OBJECTIVES: Randomised clinical trials (RCTs) are the principal source of treatment-effect evidence for health technology assessment (HTA). However, HTA decisions are often made for populations that differ from those enrolled in RCTs. When trials exclude patient groups relevant to HTA decisions - for example, those with greater comorbidity - positivity violations arise. How these violations affect transportability estimator performance is not well understood. We conducted a simulation study comparing seven estimators for transporting the average treatment effect (ATE) from a RCT to an external target population under varying degrees of positivity violation and outcome complexity.
METHODS: Data-generating mechanisms were adapted from a previously published study. Target and trial populations were simulated under a 3x2 factorial design: three selection mechanisms (medium overlap, near- and structural positivity violation) and two outcome scenarios (linear and non-linear extrapolation into non-positive region). Seven estimators were evaluated: a naïve unadjusted estimator, normalised inverse probability weighting (IPW), G-computation, standard and Hájek-normalised augmented IPW (AIPW), and standard and normalised targeted maximum likelihood estimation (TMLE). Outcome and selection models were fitted using Super Learner. Estimator performance was evaluated using bias, variance and 95% confidence interval coverage.
RESULTS: With linear outcomes, doubly robust (DR) estimators (AIPW and TMLE) and G-computation showed minimal bias across all overlap scenarios and outperformed IPW. With non-linear outcomes and medium overlap, all adjusted estimators performed well. However, under near- and structural positivity violations, all estimators, including DR estimators, exhibited substantial bias. Normalised variants of DR estimators reduced variance modestly but did not mitigate this bias. The naïve estimator was consistently the most biased.
CONCLUSIONS: DR estimators were effective for transporting treatment effects when overlap was adequate. However, no estimator was reliable under structural positivity violations when the outcome relationship was non-linear. These findings support the case for alternative approaches such as synthesising statistical and mathematical modelling in HTA decision-making.
METHODS: Data-generating mechanisms were adapted from a previously published study. Target and trial populations were simulated under a 3x2 factorial design: three selection mechanisms (medium overlap, near- and structural positivity violation) and two outcome scenarios (linear and non-linear extrapolation into non-positive region). Seven estimators were evaluated: a naïve unadjusted estimator, normalised inverse probability weighting (IPW), G-computation, standard and Hájek-normalised augmented IPW (AIPW), and standard and normalised targeted maximum likelihood estimation (TMLE). Outcome and selection models were fitted using Super Learner. Estimator performance was evaluated using bias, variance and 95% confidence interval coverage.
RESULTS: With linear outcomes, doubly robust (DR) estimators (AIPW and TMLE) and G-computation showed minimal bias across all overlap scenarios and outperformed IPW. With non-linear outcomes and medium overlap, all adjusted estimators performed well. However, under near- and structural positivity violations, all estimators, including DR estimators, exhibited substantial bias. Normalised variants of DR estimators reduced variance modestly but did not mitigate this bias. The naïve estimator was consistently the most biased.
CONCLUSIONS: DR estimators were effective for transporting treatment effects when overlap was adequate. However, no estimator was reliable under structural positivity violations when the outcome relationship was non-linear. These findings support the case for alternative approaches such as synthesising statistical and mathematical modelling in HTA decision-making.
Conference/Value in Health Info
2026-11, ISPOR Europe 2026, Vienna, Austria
Value in Health, Volume 29, Issue 12S
Code
MSR73
Topic
Epidemiology & Public Health, Health Technology Assessment, Methodological & Statistical Research
Topic Subcategory
Artificial Intelligence, Machine Learning, Predictive Analytics, Confounding, Selection Bias Correction, Causal Inference
Disease
Infectious Disease (non-vaccine), No Additional Disease & Conditions/Specialized Treatment Areas