Due to the stochastic nature of simulation based health economic models their key outcomes (e.g., total cost and QALYs per treatment, ICERs) unavoidably exhibit variance across simulation runs. Analysts and decision makers require that model results are sufficiently stable. There is no commonly accepted rule on how to measure stability and what level of stability is acceptable for the model-based decision making. An often-used approach to assess stability is charting the mean outcome versus the number of replications. Once the plotted line becomes flat, that is when there is no significant variation in the mean outcome with the increase of replications the results are considered stable. A rule of thumb is that a change of less than 1% with an extra run is acceptable. We argue that this approach is misleading, as the outcome of a repeated run with the same number of replications may result to be out of the accepted range. We propose to measure the stability of the simulation results with the standard error of the mean (SE). The advantage of this natural approach is that the SE can be estimated from a smaller sample, therefore there is no need for time consuming simulations only to determine the required number of replications. We illustrate the approach with simulated examples. The acceptable level of stability should depend on the specific decision problem. E.g., if the model predicted mean ICER is far from the decision maker’s threshold then a greater SE is acceptable, as the variance is not likely to affect the decision. For this to be evident for the decision maker, the model outcome should always be reported along with the SE, or if applicable, with the estimated probability of an outcome that would reverse the decision (e.g., as probability of cost-effectiveness for a given threshold).