Feasibility of a Voice-only Chatbot with Anthropomorphic Features Compared to a Text-based Chatbot as a Responder to Questions on Mental Health: a Randomized, Controlled Trial among Healthy Participants (Preprint)

Abstract

The use of chatbots in mental health support has increased exponentially in recent years, with studies showing that they may be effective in treating mental health problems. More recently, the use of voice-controlled visual avatars called digital humans has been introduced, which use machine learning and mimicry of facial emotions to build emotional engagement with users. It is important to study the difference in emotional response and usability preferences between text-based chatbots and digital humans for interacting with mental health services.

The aim of this study was to explore to what extent a voice-only and a text-only chatbot interface differed on usability when tested by healthy participants, using BETSY (Behavior, Emotion, Therapy System, and You) which employs two distinct interfaces: a voice-only user interface with anthropomorphic features and a text-only user interface. We also set out to explore how chatbot-generated conversations on mental health (specific to each interface) affected self-reported feelings and biometrics.

We explored to what extent a voice-only chatbot with anthropomorphic features differed from a traditional text-only chatbot regarding perception of usability though the System Usability Scale (SUS-10), emotional reactions though electroencephalography, and feeling of closeness. Healthy participants (n=45) were randomized to two groups that used a voice-only chatbot with anthropomorphic features (n=25) or a text-only chatbot with no such features (n=20). The groups were compared by linear regression analysis and t-tests.

No differences were observed between the text-only and voice-only group regarding demographic features. Mean (SD) SUS-10 score was 75.34 (10.01) [range, 57-90] for text-only chatbot versus 64.80 (14.14) (range, 40-90) for the voice-only chatbot. Both groups scored their respective chatbot interfaces as average or above average in usability. Women were more likely to report feeling annoyed by BETSY.

The text-only chatbot was perceived as significantly more user-friendly than the voice-only chatbot, although there were no significant differences in EEG measurements. Male participants exhibited lower levels of annoyance with both interfaces contrary to previously reported findings.