Understanding the Effects of Smart-Speaker-Based Surveys on Panelist Experience in Immersive Consumer Testing

Abstract

Utilizing immersive technologies to reintroduce the environmental context (i.e., visual, auditory, and olfactory cues) in sensory testing has been one area of research for improving panelist engagement. The current study sought to understand whether pairing smart-speaker questionnaires in immersive spaces could positively affect the panelist experience through enhanced ecological validity. To this end, subjects performed an immersive consumer test in which responses were collected using a traditional computer-based survey, a smart-speaker approach incorporating a direct translation of the computer questionnaire into a verbal survey requiring numeric responses, and an optimized smart-speaker survey with alternative question formatting requiring spoken word-based responses. After testing, participants answered the Engagement Questionnaire (EQ) to assess participant engagement during the test, and the System Usability Scale (SUS) survey to understand the ease, and potential adoption, of using the various survey technologies in the study. Results indicated that the traditional computer-based survey was the most engaging (p < 0.001) and usable (p < 0.001), with no differences found between the two smart-speaker surveys (p = 0.803 and p = 0.577, respectively). This suggests that the proposed optimizations for the smart-speaker surveys were not robust enough to influence engagement and usability, and further research is needed to enhance their conversational capabilities.