A Multi-Scale Temporal Convolutional Network with Attention Mechanism for Force Level Classification during Motor Imagery of Unilateral Upper-Limb Movements

Abstract

In motor imagery (MI) brain–computer interface (BCI) research, some researchers have designed MI paradigms of force under a unilateral upper-limb static state. It is difficult to apply these paradigms to the dynamic force interaction process between the robot and the patient in a brain-controlled rehabilitation robot system, which needs to induce thinking states of the patient’s demand for assistance. Therefore, in our research, according to the movement of wiping the table in human daily life, we designed a three-level-force MI paradigm under a unilateral upper-limb dynamic state. Based on the event-related de-synchronization (ERD) feature analysis of the electroencephalography (EEG) signals generated by the brain’s force change motor imagination, we proposed a multi-scale temporal convolutional network with attention mechanism (MSTCN-AM) algorithm to recognize ERD features of MI-EEG signals. Aiming at the slight feature differences of single-trial MI-EEG signals among different levels of force, the MSTCN module was designed to extract fine-grained features of different dimensions in the time–frequency domain. The spatial convolution module was then used to learn the area differences of space domain features. Finally, the attention mechanism dynamically weighted the time–frequency–space domain features to improve the algorithm’s sensitivity. The results showed that the accuracy of the algorithm was 86.4 ± 14.0% for the three-level-force MI-EEG data collected experimentally. Compared with the baseline algorithms (OVR-CSP+SVM (77.6 ± 14.5%), Deep ConvNet (75.3 ± 12.3%), Shallow ConvNet (77.6 ± 11.8%), EEGNet (82.3 ± 13.8%), and SCNN-BiLSTM (69.1 ± 16.8%)), our algorithm had higher classification accuracy with significant differences and better fitting performance.