Tracking Emotions Using an Evolutionary Model of Mental State Transitions: Introducing a New Paradigm

Abstract

Owing to rapid advancements in artificial intelligence, the role of emotion recognition has become paramount in human–computer interaction. Traditional approaches often reduce this intricate task to a mere classification problem by relying heavily on perceptual pattern-recognition techniques. However, this simplification overlooks the dynamic and multifaceted nature of human emotions. According to theories in emotion psychology, existing pattern recognition methods primarily capture external emotional expressions—termed “external emotional energy” (EEE)—rather than the nuanced underlying emotions. To address this gap, we introduce the evolutionary mental state transition model (EMSTM). In the initial phase, EMSTM employs standard pattern-recognition algorithms to extract EEE from multi-modal human expressions. Subsequently, it leverages a mental state transition network to model the dynamic transitions between emotional states, thereby predicting real-time emotions with higher fidelity. We validated the efficacy of EMSTM through experiments on 2 multi-label emotion datasets: CMU Multimodal Opinion Sentiment and Emotion Intensity (CMU-MOSEI) and Ren Chinese Emotion Corpus (Ren-CECps). The results indicate a marked improvement over conventional methods. By synergistically combining principles from psychology with computational techniques, EMSTM offers a holistic and accurate framework for real-time emotion tracking, aligning closely with the dynamic mental processes that govern human emotions.