OBJECTIVES: To compare the relative importance given to the EVIDEM criteria by Spanish (ES) and Portuguese (PT) evaluators and decision-makers involved in health technology assessment.

METHODS: Seven MCDA training sessions were performed in Spain and three in Portugal, using the EVIDEM framework. Sessions involved evaluators and decision-makers (80% pharmacist) from national, regional and local (hospital) evaluation bodies. In each session, the criteria were presented and weighted using a non-hierarchical 5-point weighting scale (5 being the highest importance). Mean score was calculated for each criterion and compared between countries applying the t-test, being the null hypothesis that Spanish weight scores would be the same as in Portugal.

RESULTS: Criteria were weighted by a total of 134 evaluators and decision makers (98 in Spain and 36 in Portugal). The most important criteria for both countries were “disease severity” and comparative efficacy”, and the less important criteria were “size of affected population” (evaluators do not choose based on number of patients to be treated), “expert consensus/clinical practice guidelines” (evaluators do not believe in guidelines because they might be biased) and “comparative non-medical costs” (evaluators feel these are not health related costs). There was no statistical difference in the assigned weight scores in 10 out of 13 criteria. Differences were observed in “size of affected population” (ES scored +0,53 compared to PT, p<0,035), “comparative patient-reported outcomes (PROs)” (ES scored -0,39 compared to PT, p<0,046) and “quality of evidence” (ES scored +0,55 compared to PT, p<0,001).

CONCLUSIONS: Evaluators and decision makers from ES and PT considered EVIDEM criteria very relevant and applicable for health technology value assessment. Overall weights assigned to criteria was very similar as well as the criteria ranking. Values assigned to “Size of affected population” and “quality of evidence” were higher in Spain, whereas “comparative PROs” were higher in Portugal.