Effect of Adding Emotion Recognition to Film Teaching—Impact of Emotion Feedback on Learning through Puzzle Films

Abstract

In this study, the scientific puzzle film, “Story of the Comet”, is taken as a case to implement scientific teaching to guide students to find correct answers, through which it can train their learning and judging abilities. The students in the experimental group received the scientific teaching guiding system of the puzzle film “Story of the Comet” with a facial emotion recognition system to recognize the emotional reaction of the subjects at the moment. According to their facial expressions of “disgust”, “sadness”, or “joy” appearing in the moment, the system presented differently captioned positive encouragement cards particularly designed for four different levels, for when the subjects answered the questions incorrectly at different levels and their emotions were detected at the same time. Furthermore, the positive encouragement cards encouraged the subjects to complete the puzzle film learning process. The subjects were students in the higher grades of Grade 5 and Grade 6 in elementary school. A total of 130 students participated in this experiment and were randomly divided into two groups. Both the control group (i.e., the group without emotion recognition) and the experiment group (i.e., the group with emotion recognition) received a before-watching test of learning effectiveness. After implementing the scientific teaching of the puzzle film “Story of the Comet”, both the control group and the experimental group also received an after-watching test of learning effectiveness. Finally, the subjects filled out a “learning satisfaction” questionnaire, “system availability” questionnaire, and “system satisfaction” questionnaire. The analysis of the results of the two groups’ tests and questionnaires: a comparative analysis of learning effectiveness indicates that there is a statistically significant difference between the choice answers of the two groups after the interactive teaching; for the experimental group, the average correct answers in the after-watching test was 5.86, which is 2.48 more than the before-watching test; that of the control group was 4.74, which is 1.47 more than the before -watching test. For comparative analysis of questionnaires for “learning satisfaction” and “system satisfaction”, the statistical data analysis indicates that the experimental group was more satisfied than the control group.