Affiliation:
1. Key Laboratory of Modern Teaching Technology, Ministry of Education, Shaanxi Normal University, Xi’an 710062, China
2. Department of Information Construction and Management, Shaanxi Normal University, Xi’an 710061, China
Abstract
In recent years, convolutional neural networks (CNNs) have played a dominant role in facial expression recognition. While CNN-based methods have achieved remarkable success, they are notorious for having an excessive number of parameters, and they rely on a large amount of manually annotated data. To address this challenge, we expand the number of training samples by learning expressions from a face recognition dataset to reduce the impact of a small number of samples on the network training. In the proposed deep joint learning framework, the deep features of the face recognition dataset are clustered, and simultaneously, the parameters of an efficient CNN are learned, thereby marking the data for network training automatically and efficiently. Specifically, first, we develop a new efficient CNN based on the proposed affinity convolution module with much lower computational overhead for deep feature learning and expression classification. Then, we develop an expression-guided deep facial clustering approach to cluster the deep features and generate abundant expression labels from the face recognition dataset. Finally, the AC-based CNN is fine-tuned using an updated training set and a combined loss function. Our framework is evaluated on several challenging facial expression recognition datasets as well as a self-collected dataset. In the context of facial expression recognition applied to the field of education, our proposed method achieved an impressive accuracy of 95.87% on the self-collected dataset, surpassing other existing methods.
Funder
National Natural Science Foundation of China
China Postdoctoral Science Foundation
Ministry of Education in China project of humanities and social sciences
Natural Science Basic Research Program of Shaanxi
Subject
Electrical and Electronic Engineering,Biochemistry,Instrumentation,Atomic and Molecular Physics, and Optics,Analytical Chemistry
Reference72 articles.
1. Deep facial expression recognition: A survey;Li;IEEE Trans. Affect. Comput.,2020
2. Face recognition: A literature review;Tolba;Int. J. Signal Process.,2006
3. Cai, Y., Li, X., and Li, J. (2023). Emotion Recognition Using Different Sensors, Emotion Models, Methods and Datasets: A Comprehensive Review. Sensors, 23.
4. Learning bases of activity for facial expression recognition;Sariyanidi;IEEE Trans. Image Process.,2017
5. Álvarez-Pato, V.M., Sánchez, C.N., Domínguez-Soberanes, J., Méndoza-Pérez, D.E., and Velázquez, R. (2020). A multisensor data fusion approach for predicting consumer acceptance of food products. Foods, 9.