OBJECTIVES: Monte Carlo simulations are driven by the generation of pseudo random numbers (PRN). Testing the effectiveness of PRN generators is rarely undertaken yet any systematic pattern or bias has implications for simulation run time and accuracy. The objective of this study was to compare two commonly used PRN in an applied setting to illustrate potential implications of low performance. METHODS: The IMS Core Diabetes Model (CDM) was used to explore their precision in detecting the onset of end stage renal disease (ESRD using the MS-Visual C++ 2008 PRN generator (MSG) and Mersenne Twister generator (MTG). One-year probabilities of ESRD for a 65 year old female smoker were generated with a systolic blood pressure (SBP) of 135 mmHG (p=0.000363) and 140 mmHg (p=0.000444). The expected one-year incidence was compared to probabilistic observations in the CDM for both PRN generators. RESULTS: The expected yearly incidence of ESRD was 0.0363% (SBP 135 mmHg) and 0.0444% (SBP (140mmHG).  Monte Carlo estimates were 0.0379% and 0.0477% using the MSG and 0.0239% for both SBP values using the MTG. The MSG overestimated expected rates by 4.41% and 7.4%; the MTG underestimated the probability of ESRD by 34.16% and 46.17% SBP for 135mmHG and 140mmHg, respectively. The deterministic relative increase in incidence of ESRD (22.3%) associated with a 5mmHg increment in SBP was similar to the MSG (25.8%); using the MTG resulted in a 0% increase in the probability of ESRD. Analysis of the frequency distribution of the MTG displayed areas sparsely populated with random variates. CONCLUSIONS: The two PRN generators tested in this analysis produced substantially different results. The differences between the two PRN algorithms were most apparent when predicting relatively rare events, such ESRD. When assessing the internal validity of Monte Carlo simulations the efficieny and robustness of PRN generators should not be assumed.