Human Perception of Social Robot’s Emotional States via Facial and Thermal Expressions

Abstract

Facial and thermal expressions can be used by humans to interpret emotions. While facial expressions can be a voluntary reaction, the change of temperature in the body is often not. Thus, a facial expression may not always be consistent with the emotional state that is expressed by the body temperature. This article aims to study the human perception of the emotional expression and the emotional state of a robot that simultaneously uses its face and body temperature. To this end, a robot, named TherMoody, which has the capability to change its body temperature from 10–55℃, was used. Then, 25 combinations (5x5: one facial expression and one thermal expression for anger, joy, fear, sadness, and neutral state) were evaluated by 15 participants. Thermal expressions were designed based on the metaphors of emotions related to temperature. The results show people tend to base their judgment of the robot’s emotional expression exclusively in its facial expression, regardless of the change of its body temperature. However, there are combinations where the thermal expression predominates over the facial expression when judging the robot’s emotional state. Thus, these combinations may produce the perception that the robot’s emotional expression is, in fact, genuine, simulated, masked, or neutralized.