Learning Status Recognition Method Based on Facial Expressions in e-Learning

Abstract

In allusion to the problem that teachers not being able to timely grasp student dynamics during online classroom, resulting in poor teaching quality, this paper proposes an online learning status analysis method that combines facial emotions with fatigue status. Specifically, we use an improved ResNet50 neural network for facial emotion recognition and quantify the detected emotions using the pleasure-arousal-dominance dimensional emotion scale. The improved network model achieved 87.51% and 75.28% accuracy on RAF-DB and FER2013 datasets, respectively, which can better detect the emotional changes of students. We use the Dlib’s face six key points detection model to extract the two-dimensional feature points of the face and judge the fatigue state. Finally, different weights are assigned to the facial emotion and fatigue state to evaluate the students’ learning status comprehensively. To verify the effectiveness of this method, experiments were conducted on the BNU-LSVED teaching quality evaluation dataset. We use this method to evaluate the learning status of multiple students and compare it with the manual evaluation results provided by expert teachers. The experiment results show that the students’ learning status evaluated using this method is basically matched with their actual status. Therefore, the classroom learning status detection method based on facial expression recognition proposed in this study can identify students’ learning status more accurately, thus realizing better teaching effect in online classroom.