OBJECTIVES: Multi-criteria decision analysis (MCDA) is a technique which is proposed for quantitative benefit/risk assessment. MCDA structures benefits and risks in a decision or value tree, either by forming hierarchical clusters of benefits and risks separately or by placing both benefits and risks on the same level allowing direct comparison of all benefits and risks (non-hierarchical). The objective of this study was to compare two approaches for structuring decision trees and to evaluate the rank-order of three TNF–ɑ inhibitors for Methotrexate type of Juvenile Idiopathic Arthritis (JIA). METHODS: The alternatives selected for evaluation were Etanercept, Infliximab and Adalimumab. Six criteria were identified for evaluation, including improvement in pain, improvement in social function, Methotrexate (MTX) discontinuation, administration reaction, serious infection and cancer. The criteria were structured both non–hierarchically and hierarchically. Two different questionnaires were developed and randomized to 42 physicians in three hospitals. Weight elicitation was performed using the Analytic Hierarchy Process.   RESULTS: In hierarchical structures, the criterion weights were more steep and the range between most and least important criterion was larger than in non-hierarchical structures. Risks were considered more important (aggregated wrisk=0.35) in an non-hierarchical structure than in an hierarchical structure (wrisk=0.20). Applying the performance weights for three TNF-a inhibitors showed that there is a different rank-order of drugs being considered. The weights elicited from non–hierarchical questionnaire showed Etanercept to be most preferred. Yet, Adalimumab was most preferred if a hierarchical structure was used. The non–hierarchical questionnaire could be transformed to a hierarchical structure after which a similar rank-order of drugs was found as in the hierarchical structure. CONCLUSIONS: The rank order of the alternatives indeed changes when the criteria were structured differently regardless where the criteria weights were obtained from similar or different group of participants. Policy implications need to be explored.