This study aimed to identify the most commonly used internal validity tests in the discrete choice experiment (DCE) literature and establish their sensitivity and specificity.
A structured literature review of recent DCE articles (2018-2020Q1) published in the health, marketing, transport economics, and environmental science literature was used to identify commonly used internal validity tests. The 2 most frequently used internal validity tests were incorporated in 4 new data collections. Respondent preferences in each application were summarized using a mixed logit model, which served as the benchmark for the subsequent sensitivity and specificity calculations. The performance of the internal validity tests was also compared with that of the root likelihood (RLH) test, which is a likelihood-based statistical validity test that is commonly used in marketing applications.
Dominant and repeated choice tasks were most commonly included in health-related DCE designs. Based on 4 applications, their specificity and sensitivity depend on the type of incorrect response pattern to be detected and on design characteristics such as the number of choice options per choice task and the number of internal validity tests as included in the experimental design. In all but one scenario, the performance of the dominant and repeated choice tasks was considerably worse than that of the RLH test.
Dominant and repeated choice tasks are unreliable screening tests and costly in terms of statistical power. The RLH test, which is a statistical test that does not require additional choice tasks to be included in the DCE design, provides a more reliable alternative.