How Many Simulations of a Dynamic Transmission Model Are Enough to Estimate Uncertainty?

Author(s)

Ahern S1, Teljeur C2, Ryan M3
1Health Information and Quality Authority, Dublin & School of Public Health, University College Cork, Cork, CO, Ireland, 2Health Information and Quality Authority, Dublin, Ireland, 3Health Information and Quality Authority & Department of Pharmacology & Therapeutics, Trinity College Dublin, Dublin, Ireland

OBJECTIVES: Dynamic transmission models (DTMs) are the preferred choice for simulating infectious diseases and the impact of associated interventions. DTMs typically incur substantial computational overheads and are therefore often estimated deterministically, affecting decision uncertainty in economic evaluations. The aim of this study was to explore the potential for using a curtailed probabilistic sensitivity analysis (PSA) in DTMs and to assess the precision associated with varying numbers of PSA simulations.

METHODS: Using a case study of a DTM developed for varicella vaccination in Ireland, the epidemiological model was run for thirty PSA simulations in the base case scenario. Four specific vaccination parameter values were drawn using Latin hypercube sampling. This was followed by additional model runs where the number of simulations ranged from one to twenty. The epidemiological model outputs on disease status for each simulation were applied to a deterministic economic model, with the outputs summarised as a net monetary benefit (NMB). The range of NMBs for the different numbers of simulations were compared to that for thirty simulations to identify the point at which the estimated imprecision stabilised.

RESULTS: While the deterministic model estimates the mean NMB accurately, it does not provide precision. The run time for the PSA simulations ranged from over five hours for one simulation to almost seven days for thirty simulations. Two to four simulations gave misleading estimates of the mean and precision (5% to 10% error). With five or more simulations, both the mean and precision were estimated with acceptable accuracy (<3% error).

CONCLUSIONS: This study highlights the considerable time resource required to run PSA simulations for a DTM of varicella vaccination. Where a limited PSA is possible (that is, with thirty or fewer simulations), fewer than five simulations can result in inaccurate estimates of the mean and or precision of the modelled outcomes.

Conference/Value in Health Info

2023-11, ISPOR Europe 2023, Copenhagen, Denmark

Value in Health, Volume 26, Issue 11, S2 (December 2023)

Code

MSR87

Topic

Economic Evaluation, Methodological & Statistical Research

Topic Subcategory

Cost-comparison, Effectiveness, Utility, Benefit Analysis

Disease

Pediatrics, Vaccines

Explore Related HEOR by Topic


Your browser is out-of-date

ISPOR recommends that you update your browser for more security, speed and the best experience on ispor.org. Update my browser now

×