Multimodal biometric fusion sentiment analysis of teachers and students based on classroom surveillance video streaming

Abstract

Abstract In the education system, teachers and students as the main body of the classroom; their emotional state in the classroom school is an important indicator of the effectiveness of the classroom. This study first explores biometric recognition, based on the needs of the classroom curriculum and the classroom monitoring as a sensor, to propose a multimodal biometric fusion detection method based on the fusion of face and gait recognition. The PCA algorithm is used to optimize the face recognition as well as the occlusion situation in the classroom to improve gait recognition, and then the face and gait are fused based on the decision layer to achieve the detection and recognition of the identity situation of teachers and students. On this basis, an expression recognition model is established using the attention mechanism, and an emotion analysis system is designed for the classroom curriculum. According to the empirical evidence of multimodal biometric fusion sentiment analysis, the mAP accuracy of this paper’s fusion method is 100% in Euclidean distance, and the accuracy is higher than 99% in cosine distance, which is obviously better than other methods, and the accuracy of this paper’s fusion recognition is above 95% under any condition limitations. At the same time, the correct rate of recognition of emotions such as listening, appreciation, resistance, doubt, and inattention are all higher than 85%, and the five indexes of average absolute error, Pearson correlation coefficient, Accuarcy5, Accuarcy2, and F12 score of this paper’s sentiment analysis have achieved the best results comparing with other sentiment analysis models, which proves the generalization and validity of this paper’s sentiment analysis.