OBJECTIVES: Different methods of meta-analysis on model parameters can lead to different outcomes of cost-effectiveness (CE) modeling. As the “true” CE is unknown, it is unclear which method performs best. We compared different methods of meta-analysis with regards to the underlying “true” CE outcome. METHODS: In a simulation study we constructed two patient populations and their treatments (“truth”): a chronic disease with events and a progressive lethal disease. We drew trials from these populations, comparing two treatments, varying the number of trials, trial sizes and between-study heterogeneity in scenarios. From each trial utilities, transition and event probabilities, risk-differences and log-risk-ratios were estimated. These parameters were synthesized using frequentist fixed-effects (FFE) and random-effects (FRE), Bayesian fixed-effects (BFE) and random-effects (BRE) models.. A CE model was filled and probabilistic sensitivity analysis was performed. We repeated this trial sampling, leading to 1000 sets of health economic outcomes for each scenario. We compared methods of meta-analysis on bias and coverage, the percentage of draws that the “true” outcome lies in the confidence interval. RESULTS: Even in the most heterogeneous scenario, biases were limited to approximately 5%, and similar for all methods, but small biases in individual treatment arms occasionally led to biases up to 30% in the difference between arms. FFE models consistently have lower coverage than BFE. With homogeneous trials, all methods have coverage above 80% for all outcomes. BRE has coverage higher than 99% for all outcomes, regardless of heterogeneity. With heterogeneity, RE methods perform better than FE and FRE has a lower coverage compared to BRE. All methods, even with heterogeneous trials, have 100% coverage around the ICER. CONCLUSIONS: BFE or BRE models are preferred in all situations, as they are more conservative. However, insight in the real level of heterogeneity is important, as using  BRE without heterogeneity will overestimate uncertainty.