OBJECTIVES:  There is a growing interest in supporting health technology assessment with multi-criteria decision analysis (MCDA). The objective of this study was to study the value of MCDA to support the advisory committee reimbursement of the Dutch National Health Care Institute. METHODS:  In a retrospective case analysis, the relevance and importance of the decision criteria and the performance of the interventions under consideration were elicited for three case studies. A mathematical MCDA was performed, criteria weights, performance scores and global values were calculated and the results were discussed with the committee members in a panel meeting. RESULTS:  The number of relevant criteria varied between nine and 14 (out of 26) between respondents (n=5), with a median number of criteria of 12. The core criteria effectiveness (17% of the total weight), burden of disease (16%) and strength of evidence (15%) were judged as most important. In the case of Pompe Disease, the disease burden (93 out of 100) and the effect size of alglucosidase alfa (91/100) were judged high, the strength of evidence was perceived as moderate (55/100) and cost-effectiveness is low (28/100). There was large variation between members in importance and performance scores. The overall value (i.e. need to reimburse) of alglucosidase alfa for Pompe Disease is judged as 76 (out of 100), 39 for the smoking cessation program and 27 for the contraceptive pill. CONCLUSIONS: Theoretically and practically, differences in value judgements between and within committee members influence the consistency and validity of recommendations across innovations. At the same time, differences in judgments are viewed as a key strength of appraisal committees. The unique and complex context in which innovations are judged in appraisal committees complicates the use of the mathematical approach to MCDA. However, the explicit deliberation of the importance of criteria, supported by facts can bring structure to the decision process.