OBJECTIVES: This study aimed to evaluate the different practical issues with determining the criteria weights of a multiple-criteria decision analysis (MCDA) framework and provide recommendations for the future based on a case-study. METHODS: An MCDA workshop was held in Kyiv, Ukraine in October 2016. The 20 participants with various healthcare-related backgrounds were asked to assign weights to five criteria (Burden of disease, Therapeutic effect, Safety, Innovation, Costs) individually, considering two decision scenarios: assessing pharmaceuticals to treat oncological diseases and rare diseases. The sum of the criteria weights was set to 100. The same participants were later assigned into 5 groups with 4 members each, to have discussions and finally agree on a set of weights that represented the joint opinion of their group. The sets of criteria weights of individuals and groups from this workshop were assessed with descriptive statistical methods. RESULTS: The means of the criteria weights of individuals and groups differed noticeably in some cases, e.g. 9.25 and 4.4 were the average values from individuals and from groups in the case of the ‘Burden of disease’ criterion for oncological drugs. The ‘Costs’ criterion had the highest standard deviation values in both decision scenarios and both in the case of individual and group-based weighting. Contrary to expectations, the smallest weight assigned a criterion was not always lower in the case of individuals than in the case of groups. Intra-group differences were also significant, e.g. one group assigned the weight of 5 to the ‘Therapeutic effect’ criterion while another group set this weight to 60. CONCLUSIONS: These findings indicate the presence of different value judgements when the same people are put in different situations. It can be recommended to consider the potential behavioural biases and limitations when constructing an MCDA framework in practice.