OBJECTIVES : Paper surveys tend to exclude out-of-the box answers and require cognitive/literate skills. In this study, participants were interviewed by a voice bot understanding natural language and compared with a paper survey containing the same questions.

METHODS : Based on the Japanese version of EQ-5D-5L, we designed a dialog flow that was implemented as an Amazon Alexa skill deployed on Echo Dot devices. The dialog featured a short-spoken tutorial, 8 demographic, 5 EQ-5D-5L health questions, and 1 question soliciting survey feedback. In order to capture most of the expected linguistic and stylistic variations, the focus was on training a robust ing state-of-the-art NLP (Natural Language Processing) techniques, and on designing a user-friendly voice dialog by 5 robust questions. After completing the voice survey participants were asked to give the same answers in a corresponding conventional EQ-5D-5L paper survey. Results of both survey methods were semi-automatically correlated and then curated by hand. A pilot study was undertaken prior to a final convenience sample of 100 participants to be performed.

RESULTS : From the pilot 22 participants completed both methods of interview. Differing responses were observed in 31.8% of the EQ-5D-5L answers given by natural language answers comparing to fixed wordings used by the EQ-5D-5L categories. With the paper survey as the baseline, 93.6% of the voice survey answers were classified correctly and 6.4% of all answers did not match any of the 5 levels of difficulty.

CONCLUSIONS : Voice technology can make surveys accessible to currently excluded target groups broadening participation, whilst accommodating their desire to express nuances and out-of-box answers that do not fit the rigid standard categories of a paper survey, more participants to supply additional new thoughts and dimensions that were not foreseen by the creators of the survey whilst responding to questions.