Are Discrete Choice Experiment Results Robust to Omitted Attributes? Evidence From a Vaccine Study
Author(s)
Mesfin Genie, PhD1, Verity Watson, BA, MS, PhD2, Stephane Luchini, PhD3, Michael Schwarzinger, PhD, MD4.
1Duke University, Newcastle, Australia, 2Senior Research Economist, RTI Health Solutions, Manchester, United Kingdom, 3Université Aix Marseille, Marseille, France, 4Bordeaux University Hospital, Paris, France.
1Duke University, Newcastle, Australia, 2Senior Research Economist, RTI Health Solutions, Manchester, United Kingdom, 3Université Aix Marseille, Marseille, France, 4Bordeaux University Hospital, Paris, France.
OBJECTIVES: In health care product development, innovations take time and new information during product development and post-approval means that knowledge about a product’s benefits and risks evolves. Discrete choice experiments (DCEs) are often developed pre-approval when scientific evidence is emerging and before products are in routine use. Understanding whether individuals’ preferences elicited for evolving products are robust to changing knowledge is important.
METHODS: A recent example of rapidly emerging knowledge was the evolution of vaccines against COVID-19. Shortly after the COVID-19 vaccines were developed information about the duration of immunity and safety became available. We conducted a preference study for vaccination against COVID-19 in December 2020 and fielded two versions of a DCE survey (BASE and EXTRA) that differed with respect to the included attributes (EXTRA included booster vaccination) and the levels of the vaccine safety attribute (EXTRA included more levels). We randomly allocate respondents across the two versions and the DCE was identical in all other respects. We test the effect of this additional/omitted attribute on the relative attribute importance for the remaining attributes and on choice consistency and test heterogeneity of the impact on choice consistency across the sample.
RESULTS: The introduction of an additional attribute did not change the underlying preference structure, as measured by relative attribute importance across both DCEs. However, the additional attribute the and the additional attribute levels reduced consistency in respondents’ choices, on average. The effect of extra information varied across respondents: for respondents who were vaccine hesitant extra information reduced choice consistency; however, for respondents who were vaccine willing the extra information improved consistency.
CONCLUSIONS: Our results show that DCE study results are robust to the inclusion/exclusion of relevant attributes, but including more attributes may lower the precision of estimated preferences
METHODS: A recent example of rapidly emerging knowledge was the evolution of vaccines against COVID-19. Shortly after the COVID-19 vaccines were developed information about the duration of immunity and safety became available. We conducted a preference study for vaccination against COVID-19 in December 2020 and fielded two versions of a DCE survey (BASE and EXTRA) that differed with respect to the included attributes (EXTRA included booster vaccination) and the levels of the vaccine safety attribute (EXTRA included more levels). We randomly allocate respondents across the two versions and the DCE was identical in all other respects. We test the effect of this additional/omitted attribute on the relative attribute importance for the remaining attributes and on choice consistency and test heterogeneity of the impact on choice consistency across the sample.
RESULTS: The introduction of an additional attribute did not change the underlying preference structure, as measured by relative attribute importance across both DCEs. However, the additional attribute the and the additional attribute levels reduced consistency in respondents’ choices, on average. The effect of extra information varied across respondents: for respondents who were vaccine hesitant extra information reduced choice consistency; however, for respondents who were vaccine willing the extra information improved consistency.
CONCLUSIONS: Our results show that DCE study results are robust to the inclusion/exclusion of relevant attributes, but including more attributes may lower the precision of estimated preferences
Conference/Value in Health Info
2025-11, ISPOR Europe 2025, Glasgow, Scotland
Value in Health, Volume 28, Issue S2
Code
PCR18
Topic
Epidemiology & Public Health, Methodological & Statistical Research, Patient-Centered Research
Disease
Vaccines