The Importance of Specifying the Estimand in Meta-Analyses in the Presence of Treatment Switching
Author(s)
Quang Vuong, MSc1, Rebecca Metcalfe, BA, MA, PhD2, Antonio Remiro Azócar, PhD3, Anders Gorst-Rasmussen, MSc, PhD4, Oliver Keene, MSc5, Jay J. Park, PhD6;
1Core Clinical Sciences, Vancouver, BC, Canada, 2Centre for Advancing Health Outcomes, Research Methodologist, Vancouver, BC, Canada, 3Novo Nordisk Pharma, Methods & Outreach, Madrid, Spain, 4Novo Nordisk A/S, Søborg, Denmark, 5KeeneONStatistics, Maidenhead, United Kingdom, 6McMaster University, Department of Health Research Methodology, Evidence, and Impact, Hamilton, ON, Canada
1Core Clinical Sciences, Vancouver, BC, Canada, 2Centre for Advancing Health Outcomes, Research Methodologist, Vancouver, BC, Canada, 3Novo Nordisk Pharma, Methods & Outreach, Madrid, Spain, 4Novo Nordisk A/S, Søborg, Denmark, 5KeeneONStatistics, Maidenhead, United Kingdom, 6McMaster University, Department of Health Research Methodology, Evidence, and Impact, Hamilton, ON, Canada
OBJECTIVES: The ICH E9(R1) addendum describes the estimands framework which emphasizes the importance of reporting strategies to account for intercurrent events (ICEs) in clinical trials. However, the implications of the estimands framework for meta-analysis have not been well studied. In the context of treatment switching as an ICE, we examined the bias caused by pooling together estimates targeting different estimands in a meta-analysis of randomized clinical trials (RCTs).
METHODS: We simulated overall survival data for eight RCTs that allowed patients in the control group to switch to the intervention treatment after disease progression. For each RCT, we estimated a treatment policy estimand that ignored treatment switching, and a hypothetical estimand that censored treatment switchers at the time of switching. Then, we pooled together RCT effect estimates under fixed-effects and random-effects meta-analytical models while varying the proportions of treatment policy and hypothetical effect estimates. We examined the bias of effect estimates from meta-analyses that pooled different types of effect estimates vs those that pooled only treatment policy or hypothetical estimates.
RESULTS: On average, pooling purely hypothetical estimates produced treatment effects that were further from the null than pooling purely treatment policy estimates. This was true across different allocation ratios and control arm treatment switching rates. The rate of concordance between the estimates pooled and the target estimand directly impacted bias and coverage, with less concordance yielding greater bias and poorer coverage.
CONCLUSIONS: We found that pooling estimates targeting different estimands results in meta-analytic estimators that reflect neither the treatment policy estimand nor the hypothetical estimand. This finding suggests that pooling estimates of varying target estimands even under a random-effects model can produce misleading results. Adopting the estimands framework for meta-analysis may improve alignment between meta-analytic results and the clinical research question of interest.
METHODS: We simulated overall survival data for eight RCTs that allowed patients in the control group to switch to the intervention treatment after disease progression. For each RCT, we estimated a treatment policy estimand that ignored treatment switching, and a hypothetical estimand that censored treatment switchers at the time of switching. Then, we pooled together RCT effect estimates under fixed-effects and random-effects meta-analytical models while varying the proportions of treatment policy and hypothetical effect estimates. We examined the bias of effect estimates from meta-analyses that pooled different types of effect estimates vs those that pooled only treatment policy or hypothetical estimates.
RESULTS: On average, pooling purely hypothetical estimates produced treatment effects that were further from the null than pooling purely treatment policy estimates. This was true across different allocation ratios and control arm treatment switching rates. The rate of concordance between the estimates pooled and the target estimand directly impacted bias and coverage, with less concordance yielding greater bias and poorer coverage.
CONCLUSIONS: We found that pooling estimates targeting different estimands results in meta-analytic estimators that reflect neither the treatment policy estimand nor the hypothetical estimand. This finding suggests that pooling estimates of varying target estimands even under a random-effects model can produce misleading results. Adopting the estimands framework for meta-analysis may improve alignment between meta-analytic results and the clinical research question of interest.
Conference/Value in Health Info
2025-05, ISPOR 2025, Montréal, Quebec, CA
Value in Health, Volume 28, Issue S1
Code
MSR92
Topic
Methodological & Statistical Research
Disease
SDC: Oncology