THE OPTIMAL NUMBER OF MONTE CARLO SIMULATIONS TO BE PERFORMED IN PROBABILISTIC SENSITIVITY ANALYSIS- EMPIRICAL EVIDENCE FROM ECONOMIC MODELS CONSTRUCTED FOR SUBMISSION TO THE NATIONAL INSTITUTE FOR HEALTH AND CLINICAL EXCELLENCE

Author(s)

Batty AJ1, Paulden M21BresMed Health Solutions, Sheffield, United Kingdom, 2University of Toronto, Toronto, ON, Canada

OBJECTIVES: Probabilistic Sensitivity Analysis (PSA) is a common technique to assess uncertainty in economic models, with the majority of economic model publications now containing some form of PSA. Historically the number of simulations performed has been set at arbitrary levels (e.g. 1,000 simulations), however the aim of this research is to rationalise the number of simulations performed, minimising both wasted computational time and the risk of incorrect conclusions being drawn. METHODS: The analysis investigates the number of simulations required in order for a Cost-Effectiveness Acceptability Curve (CEAC) to remain stable at the periphery. Secondary analyses focused on the number of simulations required to give reliable estimates of the mean values in these (non-linear) models. In the UK the National Institute for Health and Clinical Excellence (NICE) require manufacturers to submit PSA as part of the Single Technology Appraisal (STA) process. Models from different contract research agencies that have been constructed in Microsoft Excel, for submission to NICE, were then used to generate 50,000 simulations per model.  This data was retrospectively analysed to determine the number of simulations required such that a cost-effectiveness acceptability curve would remain stable at the periphery (5th and 95th percentiles). Secondary analyses focused on the number of simulations required to give reliable estimates of the mean values in these (non-linear) models. RESULTS: Preliminary analyses suggest that conventional numbers of simulations are sufficient to estimate the CEAC at low levels of precision at the 5% and 95% limits and generate the mean value. However this is not the case if high levels of precision are required. CONCLUSIONS: Research in to the optimum number of Monte Carlo Simulations allows analysts to ground the number performed in empirical data, and suggests that accuracy can be achieved without spurious precision, wasted computing time, or worse, unreliable / unstable conclusions.

Conference/Value in Health Info

2010-11, ISPOR Europe 2010, Prague, Czech Republic

Value in Health, Vol. 13, No. 7 (November 2010)

Code

PMC20

Topic

Methodological & Statistical Research

Topic Subcategory

Modeling and simulation

Disease

Multiple Diseases

Explore Related HEOR by Topic


Your browser is out-of-date

ISPOR recommends that you update your browser for more security, speed and the best experience on ispor.org. Update my browser now

×