Abstract
Owing to the availability of a wide range of emotion recognition applications in our lives, such as for mental status calculation, the demand for high-performance emotion recognition approaches remains uncertain. Nevertheless, the wearing of facial masks has been indispensable during the COVID-19 pandemic. In this study, we propose a graph-based emotion recognition method that adopts landmarks on the upper part of the face. Based on the proposed approach, several pre-processing steps were applied. After pre-processing, facial expression features need to be extracted from facial key points. The main steps of emotion recognition on masked faces include face detection by using Haar–Cascade, landmark implementation through a media-pipe face mesh model, and model training on seven emotional classes. The FER-2013 dataset was used for model training. An emotion detection model was developed for non-masked faces. Thereafter, landmarks were applied to the upper part of the face. After the detection of faces and landmark locations were extracted, we captured coordinates of emotional class landmarks and exported to a comma-separated values (csv) file. After that, model weights were transferred to the emotional classes. Finally, a landmark-based emotion recognition model for the upper facial parts was tested both on images and in real time using a web camera application. The results showed that the proposed model achieved an overall accuracy of 91.2% for seven emotional classes in the case of an image application. Image based emotion detection of the proposed model accuracy showed relatively higher results than the real-time emotion detection.
Funder
Korea Agency for Technology and Standards
Gachon University
Subject
Electrical and Electronic Engineering,Biochemistry,Instrumentation,Atomic and Molecular Physics, and Optics,Analytical Chemistry
Reference63 articles.
1. Recognizing action units for facial expression analysis;Tian;IEEE Trans. Pattern Anal. Mach. Intell.,2001
2. A lightweight convolutional neural network for real-time facial expression detection;Zhou;IEEE Access,2020
3. Joint facial expression recognition and intensity estimation based on weighted votes of image sequences;Kamarol;Pattern Recognit. Lett.,2017
4. Cai, J., Chang, O., Tang, X.-L., Xue, C., and Wei, C. Facial Expression Recognition Method Based on Sparse Batch Normalization CNN. Proceedings of the 2018 37th Chinese Control Conference (CCC).
5. Emotion Recognition From Facial Expressions and Its Control Using Fuzzy Logic;Chakraborty;IEEE Trans. Syst. Man Cybern.-Part A Syst. Humans,2009
Cited by
22 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献