OBJECTIVES: Swing-weighting (SW) and discrete choice experiments (DCEs) are two possible techniques for developing scoring functions in the context of multi-criteria decision analysis (MCDA). The objective of this study is to compare estimates from both methods using a simulation study.

METHODS: A fictitious utility model representing preferences of a population among 5 attributes, each with 3 levels, was defined as the sum of marginal utilities over attributes. The model allowed for variability around marginal utilities between people and included a random term capturing aspects of the utility not related to the 5 attributes. The DCE was assumed to be conducted among 200 individuals, using an orthogonal balanced design with 18 choice tasks. Responses to choices were simulated randomly, based on probabilities calculated as a logit function of utilities. The SW was assumed to be conducted among 5 to 50 individuals. The correlation was calculated between the estimated utility scores from SW, the predicted utility scores from the DCE model and the underlying “true” utility.

RESULTS: The correlation between DCE and true utilities was approximately 0.93. The correlation between SW score and true utility ranged between 0.56 and 0.97 for each individual. The SW score based on average responses from 5 individuals had a correlation of 0.85 with true utility. The correlation of SW with the true utility became equal to DCE after averaging responses over 30 or more individuals.

CONCLUSIONS: SW and DCE scores both correlate relatively well with true utility. A DCE with 18 choice tasks and 200 individuals performed as well as a SW with 30 individuals when similar levels of error were assumed. This simulation framework could be applied when deciding which technique to use when planning a MCDA.