REPRODUCIBILITY GAPS IN A PUBLISHED HERPES ZOSTER VACCINE ECONOMIC EVALUATION: A STRUCTURED RECONSTRUCTION USING A WEB-BASED HTA APPLICATION
Author(s)
Eemil Karttunen, MSc1, Ville Kainu, MD, PhD, Executive MBA2, Simo Jääskeläinen, MSc3, Mikko Kosunen, MSc2.
1EasyHTA Oy, Helsinki, Finland, 2Pfizer Oy, Helsinki, Finland, 3Medaffcon Oy, Espoo, Finland.
1EasyHTA Oy, Helsinki, Finland, 2Pfizer Oy, Helsinki, Finland, 3Medaffcon Oy, Espoo, Finland.
OBJECTIVES: To identify reproducibility gaps when independently reconstructing a published economic evaluation of herpes zoster vaccination, conducted by Finland's Institute for Health and Welfare to inform national immunization decisions.
METHODS: We reconstructed a published evaluation from publicly available documentation with a novel web-based vaccine modelling application, capturing structure, parameters, and assumptions, and documenting missing or unclear information. Each element was classified as directly reproducible, traceable but requiring interpretation, requiring back-calculation, or not identifiable. We validated the oldest cohort (shortest time horizon, least sensitive to unverified inputs), defining effective reproduction as results within 5% of the published incremental cost-effectiveness ratio (ICER), with baseline incidence calibrated to reported case counts.
RESULTS: The decision problem, intervention, comparator, and modelling approach were fully identifiable. Of 15 extracted elements, 12 were directly reproducible. Population size was unreported, required interpretation and was inferred from national projections; intermediate outputs were insufficiently disaggregated. Cost per event required reconstruction and back-calculation from complication costs and frequencies. Baseline incidence could neither be located nor reconstructed. Efficacy, coverage, mortality, utilities, and costs were sufficiently transparent. The calibrated incidence matched no reported population or subgroup, suggesting an undocumented adjustment. With incidence calibrated to reported cases, the reconstructed ICER for the oldest cohort (85 years) independently matched the published estimate within 5% (4.1%; €84,203 vs €87,800/QALY). Younger cohorts were not validated, as the calibration's influence grows with horizon length and age-band span.
CONCLUSIONS: Novel tools can accelerate and strengthen vaccine economic evaluations, improving transparency and traceability, but cannot overcome incomplete reporting. This case study shows that analyses may inform decisions while remaining only partly reproducible. When non-transparent evaluations also lead to price recommendations in relation to willingness-to-pay thresholds, concerns about the public process arise. More complete reporting of model inputs, intermediate calculations, and parameter derivations would improve auditability, confidence, and reuse of vaccine HTA evidence for decision-making.
METHODS: We reconstructed a published evaluation from publicly available documentation with a novel web-based vaccine modelling application, capturing structure, parameters, and assumptions, and documenting missing or unclear information. Each element was classified as directly reproducible, traceable but requiring interpretation, requiring back-calculation, or not identifiable. We validated the oldest cohort (shortest time horizon, least sensitive to unverified inputs), defining effective reproduction as results within 5% of the published incremental cost-effectiveness ratio (ICER), with baseline incidence calibrated to reported case counts.
RESULTS: The decision problem, intervention, comparator, and modelling approach were fully identifiable. Of 15 extracted elements, 12 were directly reproducible. Population size was unreported, required interpretation and was inferred from national projections; intermediate outputs were insufficiently disaggregated. Cost per event required reconstruction and back-calculation from complication costs and frequencies. Baseline incidence could neither be located nor reconstructed. Efficacy, coverage, mortality, utilities, and costs were sufficiently transparent. The calibrated incidence matched no reported population or subgroup, suggesting an undocumented adjustment. With incidence calibrated to reported cases, the reconstructed ICER for the oldest cohort (85 years) independently matched the published estimate within 5% (4.1%; €84,203 vs €87,800/QALY). Younger cohorts were not validated, as the calibration's influence grows with horizon length and age-band span.
CONCLUSIONS: Novel tools can accelerate and strengthen vaccine economic evaluations, improving transparency and traceability, but cannot overcome incomplete reporting. This case study shows that analyses may inform decisions while remaining only partly reproducible. When non-transparent evaluations also lead to price recommendations in relation to willingness-to-pay thresholds, concerns about the public process arise. More complete reporting of model inputs, intermediate calculations, and parameter derivations would improve auditability, confidence, and reuse of vaccine HTA evidence for decision-making.
Conference/Value in Health Info
2026-11, ISPOR Europe 2026, Vienna, Austria
Value in Health, Volume 29, Issue 12S
Code
MSR273
Topic
Methodological & Statistical Research
Disease
Vaccines