Multi-Region and Multi-Band Electroencephalogram Emotion Recognition Based on Self-Attention and Capsule Network

Abstract

Research on emotion recognition based on electroencephalogram (EEG) signals is important for human emotion detection and improvements in mental health. However, the importance of EEG signals from different brain regions and frequency bands for emotion recognition is different. For this problem, this paper proposes the Capsule–Transformer method for multi-region and multi-band EEG emotion recognition. First, the EEG features are extracted from different brain regions and frequency bands and combined into feature vectors which are input into the fully connected network for feature dimension alignment. Then, the feature vectors are inputted into the Transformer for calculating the self-attention of EEG features among different brain regions and frequency bands to obtain contextual information. Finally, utilizing capsule networks captures the intrinsic relationship between local and global features. It merges features from different brain regions and frequency bands, adaptively computing weights for each brain region and frequency band. Based on the DEAP dataset, experiments show that the Capsule–Transformer method achieves average classification accuracies of 96.75%, 96.88%, and 96.25% on the valence, arousal, and dominance dimensions, respectively. Furthermore, in emotion recognition experiments conducted on individual brain regions or frequency bands, it was observed that the frontal lobe exhibits the highest average classification accuracy, followed by the parietal, temporal, and occipital lobes. Additionally, emotion recognition performance is superior for high-frequency band EEG signals compared to low-frequency band signals.