Improvement of emotion recognition methods based on neural networks

Abstract

This article analyzes the use of microexpressions – subtle facial movements that are difficult for the human eye to notice, and even more difficult to immediately analyze, even specialists in the field do not always succeed in this perfectly, because their speed is only 1/5 to 1/3 of a second, for assessment of psychological state using artificial intelligence methods. The research is aimed at improving the analysis of micro-mimicry for accurate identification of emotions and psychological state. An overview of implemented technological solutions based on CNN was conducted, and a method for their improvement was found. An experimental test conducted on video recordings of people experiencing various emotions showed the high accuracy of the developed method in recognizing emotions and psychological state. Despite the challenges of the scarcity of microexpression datasets and the subtlety of facial movements, the paper presents a CapsuleNet model for microexpression recognition, builds a system architecture, and conducts testing. By combining three main data sets (SMIC, CASME II and SAMM) into a unified cross-database, the method developed in the work tests the possibility of generalization of the model by different subject characteristics. The performance of CapsuleNet, evaluated by cross-baseline benchmarking and Leave-One-Object-Out validation, significantly outperforms the baseline (LBP-TOP) and other improved of an CNN models. The paper shows that the performance of the developed model, determined by unweighted average recall and F1 scores, outperforms both the LBP-TOP baseline and other state-of-the-art CNN models. In a comprehensive microexpression recognition system. First, we process the data to identify the peak frames in the sequences and isolate the face region in these frames. These processed face images are then moved to CapsuleNet for the classification. The results of the work is to develop and complement methods of emotional artificial intelligence, offering new insights into micromimic assessment of psychological states that affect mental health, human-computer interaction, and social robotics. This technology has potential for development and expansion. This is an additional opportunity for companies that work with people and it is important for them to monitor their productivity, as it is directly related to the psychological state.