A Novel Lightweight Deep Convolutional Neural Network Model for Human Emotions Recognition in Diverse Environments

Abstract

A picture is worth a thousand words to convey information during daily life communication. Recognizing a person’s emotions from facial expressions (FEs) has become a prevalent research field in the past few decades. Deep learning (DL) models, mainly deep convolutional neural networks (DCNNs), are trending in the last few years to perform recognition/classification tasks. Several prior DCNNs have resulted in good recognition accuracy for facial emotion recognition (FER) systems. Still, there is a need for an effective as well as efficient FER system that can recognize the FEs irrespective of the illumination conditions, subjects’ gender, age range, geographical locations, race, etc. In this work, we have presented a novel lightweight DCNN model that can recognize the FEs in the aforementioned conditions. We have made it lightweight by optimal selection of its hidden layers, which ultimately resulted in the reduced number of floating point operations (FLOPs). We have embedded the data augmentation step into the model’s training phase to enhance its generalization ability. An early stopping criterion is also introduced to prevent the model’s overfitting. We have trained and evaluated the performance of the proposed model on widely used benchmarks for FEs databases. We have selected five diverse databases (two collected in the lab, one based on stylized cartoon characters, and two collections in an unconstrained realistic environment), including CK+, Karolinska Directed Emotional Faces (KDEF), Facial Expression Research Group (FERG), Facial Expression Recognition-2013 (FER-2013), and Real-World Affective Faces Database (RaF-DB). The recognition accuracy of 99.98%, 99.25%, 88.17%, 84.09%, 69.87%, and 69.16% is achieved for the FERG, CK+, KDEF, RaF-DB, FER-2013 human faces (FER-2013H), and FER-2013 complete (FER-2013C) database, respectively. Our model outperforms the number of state-of-the-art FER approaches in terms of recognition accuracy and FLOPs.