WHEN GUIDANCE MEETS REALITY: LESSONS FROM RECENT HTA SUBMISSIONS IN REPORTING EFFECT MODIFIERS
Author(s)
Florint Kroi, MSc1, Bente van Amsterdam, MSc1, Peter Wigfield, MSc1, Victor Laliman-Khara, MSc2.
1Cytel Inc., Rotterdam, Netherlands, 2Cytel Inc., Thompson, MB, Canada.
1Cytel Inc., Rotterdam, Netherlands, 2Cytel Inc., Thompson, MB, Canada.
OBJECTIVES: To assess how effect modifiers (EMs) are identified, evaluated, and incorporated in health technology assessments (HTAs), focusing on appraisals by the National Institute for Health and Care Excellence (NICE), the Canadian Drug Agency (CDA), and the Zorginstituut Nederland (ZIN), as well as their respective methodological guidelines. Additionally, the study aimed to identify key methodological gaps relevant to population‑adjusted indirect comparisons (PAICs).
METHODS: A review of oncology HTA submissions from NICE, CDA, and ZIN was conducted including submissions from 2021 to May 2026, alongside relevant methodological guidance (including European Joint Clinical Assessment). Evidence was extracted on EM identification, selection processes, application in PAIC methods, credibility assessment, and incorporation into economic models. HTA critiques were systematically mapped to the Instrument to Assess the Credibility of Effect Modification Analyses (ICEMAN) domains to develop an HTA-aligned framework for EM justification.
RESULTS: A total of 31, 23 and 1 submissions including PAICs were identified for NICE, CDA and ZIN, respectively. Critiques were summarized across five dimensions of the ICEMAN framework. While PAIC reporting by ZIN remained limited, NICE and CDA demonstrated more extensive use of PAICs, with covariates typically selected based on clinical plausibility and data availability. Critiques were consistent across agencies, including absence of formal EM selection frameworks, unclear covariate sufficiency, lack of distinction between prognostic factors and EMs, and absence of structured credibility assessment. EMs were not explicitly modelled, limiting their integration into statistical analyses and economic models. Residual confounding reduced effective sample size, and violations of the transitivity assumption were frequently highlighted, particularly in unanchored and real-world evidence based comparisons.
CONCLUSIONS: HTA practice recognizes the importance of EMs but lacks a structured framework linking identification, validation, and application. An ICEMAN approach can improve transparency and consistency to meet guideline requirements. However, further research is needed to address data limitations, particularly in non-randomized controlled trial settings.
METHODS: A review of oncology HTA submissions from NICE, CDA, and ZIN was conducted including submissions from 2021 to May 2026, alongside relevant methodological guidance (including European Joint Clinical Assessment). Evidence was extracted on EM identification, selection processes, application in PAIC methods, credibility assessment, and incorporation into economic models. HTA critiques were systematically mapped to the Instrument to Assess the Credibility of Effect Modification Analyses (ICEMAN) domains to develop an HTA-aligned framework for EM justification.
RESULTS: A total of 31, 23 and 1 submissions including PAICs were identified for NICE, CDA and ZIN, respectively. Critiques were summarized across five dimensions of the ICEMAN framework. While PAIC reporting by ZIN remained limited, NICE and CDA demonstrated more extensive use of PAICs, with covariates typically selected based on clinical plausibility and data availability. Critiques were consistent across agencies, including absence of formal EM selection frameworks, unclear covariate sufficiency, lack of distinction between prognostic factors and EMs, and absence of structured credibility assessment. EMs were not explicitly modelled, limiting their integration into statistical analyses and economic models. Residual confounding reduced effective sample size, and violations of the transitivity assumption were frequently highlighted, particularly in unanchored and real-world evidence based comparisons.
CONCLUSIONS: HTA practice recognizes the importance of EMs but lacks a structured framework linking identification, validation, and application. An ICEMAN approach can improve transparency and consistency to meet guideline requirements. However, further research is needed to address data limitations, particularly in non-randomized controlled trial settings.
Conference/Value in Health Info
2026-11, ISPOR Europe 2026, Vienna, Austria
Value in Health, Volume 29, Issue 12S
Code
MSR85
Topic
Clinical Outcomes, Health Technology Assessment, Methodological & Statistical Research
Topic Subcategory
Confounding, Selection Bias Correction, Causal Inference
Disease
No Additional Disease & Conditions/Specialized Treatment Areas, Oncology