Fusion of multi representation and multi descriptors for facial expression recognition

Abstract

Abstract Facial Expression Recognition has become vital for efficient Human Computer Interaction. In this paper, we propose effective facial expression recognition approachfor recognizing six basic facial expressions. Our approach consists of three main phases which are: (1) face detection and pre-processing, (2) features extraction and (3) facial expression classification. The face pre-processing phase is performed using the facial landmarks. After the face is aligned and cropped, facial regions of interest (eyes, nose and mouth) are detected. In the features extraction phase, we used Histogram of oriented gradients (HOG), Local Binary Pattern (LBP) and the fusion of the two features. For the last step, Support Vector Machine (SVM) is used to recognize the facial expression. To evaluate the performance of our approach, we used three popular datasets which are The Extended Cohn-Kanade (CK+), The Japanese Female Facial Expression (JAFFE) and Oulu-CASIA NIR-VIS dataset (CASIA), In addition, 10 folds cross-validation scheme is used to evaluate the performance of our approach. Our proposed fusion of multi representations and multi descriptors achieves better or competitive performance compared with the state-of-the-art methods. The accuracies of our approach are 99.18%, 95.77% and 99.09% for CK+, JAFFE and CASIA, respectively. The results prove the efficiency of our approach although the challenging conditions from one dataset to another.