Between Reality and Delusion: Challenges of Applying Large Language Models to Companion Robots for Open-Domain Dialogues with Older Adults

Abstract

AbstractThis work aims to provide initial guidelines towards developing companion robots with large language models (LLMs) to be part of everyday lives of older adults. Using iterative participatory design (co-design) approaches, we analyze the challenges of applying LLMs for multi-modal open-domain dialogue, deriving from older adults' (one-to-one) interactions with a personalized companion robot, built on Furhat robot with GPT-3.5. An initial study with 6 Swedish-speaking older adults (65 and older) showed that the robot frequently interrupted the users, responded slowly and repetitively, engaged in superficial conversations, and caused a barrier in the interaction due to foreign language (English). Upon incremental technical developments to address these issues, participatory design workshops were conducted with 28 Swedish-speaking older adults. While the interactions (in Swedish) were smoother, less disrupted, and more varied in topics and responses, further challenges were observed due to hallucinations and obsolete information, and disengagement cues, causing frustration, confusion, and worry.