A Novel Fuzzy Optimized CNN-RNN Method for Facial Expression Recognition

Abstract

Facial expression is one of the important ways of transferring emotion in interpersonal communication, and it has been widely used in many interpersonal communication systems. The traditional facial expression recognition methods are not intelligent enough to manage the model uncertainty. The deep learning method has obvious ability to deal with model uncertainty in the image recognition. The deep learning method is able to complete the facial expression work, but the recognition rate can be further improved by a hybrid learning strategy. In this paper, a Fuzzy optimized convolutional neural network-recurrent neural network (CNN-RNN) method for facial expression recognition is proposed to solve the problems of direct image convolution without image enhancement and simple convolution stack ignoring feature layer-by-layer convolution resulting in information loss. Firstly, each face image is scaled by the bilinear interpolation and the affine transformation is adopted to expand the image data to avoid the shortage of the data set. Then the feature map of the facial expression is extracted by the CNN with small information loss. To deal with the uncertainty in the feature map, the Fuzzy logic is employed to reduce the uncertainty by recognizing the highly nonlinear relationship between the features. Then the output of the Fuzzy model is fed with the RNN to classify different facial expression images. The recognition results based on the open datasets CK, Jaffe, and FER2013 show that the proposed Fuzzy optimized CNN-RNN method has a certain improvement in the recognition effect of different facial expression data sets compared with current popular algorithms.