Spatial–temporal features-based EEG emotion recognition using graph convolution network and long short-term memory

Abstract

Abstract Objective. Emotion recognition on the basis of electroencephalography (EEG) signals has received a significant amount of attention in the areas of cognitive science and human–computer interaction (HCI). However, most existing studies either focus on one-dimensional EEG data, ignoring the relationship between channels, or only extract time–frequency features while not involving spatial features. Approach. We develop spatial–temporal features-based EEG emotion recognition using a graph convolution network (GCN) and long short-term memory (LSTM), named ERGL. First, the one-dimensional EEG vector is converted into a two-dimensional mesh matrix, so that the matrix configuration corresponds to the distribution of brain regions at EEG electrode locations, thus to represent the spatial correlation between multiple adjacent channels in a better way. Second, the GCN and LSTM are employed together to extract spatial–temporal features; the GCN is used to extract spatial features, while LSTM units are applied to extract temporal features. Finally, a softmax layer is applied to emotion classification. Main results. Extensive experiments are conducted on the A Dataset for Emotion Analysis using Physiological Signals (DEAP) and the SJTU Emotion EEG Dataset (SEED). The classification results of accuracy, precision, and F-score for valence and arousal dimensions on DEAP achieved 90.67% and 90.33%, 92.38% and 91.72%, and 91.34% and 90.86%, respectively. The accuracy, precision, and F-score of positive, neutral, and negative classifications reached 94.92%, 95.34%, and 94.17%, respectively, on the SEED dataset. Significance. The above results demonstrate that the proposed ERGL method is encouraging in comparison to state-of-the-art recognition research.