Personal Resilience Can Be Well Estimated from Heart Rate Variability and Paralinguistic Features during Human–Robot Conversations

Abstract

Mental health is as crucial as physical health, but it is underappreciated by mainstream biomedical research and the public. Compared to the use of AI or robots in physical healthcare, the use of AI or robots in mental healthcare is much more limited in number and scope. To date, psychological resilience—the ability to cope with a crisis and quickly return to the pre-crisis state—has been identified as an important predictor of psychological well-being but has not been commonly considered by AI systems (e.g., smart wearable devices) or social robots to personalize services such as emotion coaching. To address the dearth of investigations, the present study explores the possibility of estimating personal resilience using physiological and speech signals measured during human–robot conversations. Specifically, the physiological and speech signals of 32 research participants were recorded while the participants answered a humanoid social robot’s questions about their positive and negative memories about three periods of their lives. The results from machine learning models showed that heart rate variability and paralinguistic features were the overall best predictors of personal resilience. Such predictability of personal resilience can be leveraged by AI and social robots to improve user understanding and has great potential for various mental healthcare applications in the future.