Enhanced Deep Learning Hybrid Model of CNN Based on Spatial Transformer Network for Facial Expression Recognition

Abstract

One of the most common approaches through which people communicate is facial expressions. A large number of features documented in the literature were created by hand, with the goal of overcoming specific challenges such as occlusions, scale, and illumination variations. These classic methods are then applied to a dataset of facial images or frames in order to train a classifier. The majority of these studies perform admirably on datasets of images shot in a controlled environment, but they struggle with more difficult datasets (FER-2013) that have higher image variation and partial faces. The nonuniform features of the human face as well as changes in lighting, shadows, facial posture, and direction are the key obstacles. Techniques of deep learning have been studied as a set of methodologies for gaining scalability and robustness on new forms of data. In this paper, we look at how well-known deep learning techniques (e.g. GoogLeNet, AlexNet) perform when it comes to facial expression identification, and propose an enhanced hybrid deep learning model based on STN for facial emotion recognition, which gives the best feature extraction and classification in one go and maximizes the accuracy for a large number of samples on FERG, JAFFE, FER-2013, and CK+ datasets. It is capable of focusing on the main parts of the face and attaining extensive development over preceding fashions on the FERG, JAFFE, CK+ datasets, and the more challenging one namely FER-2013.