A Joint Mapping and Synthesis Approach for Multiview Facial Expression Recognition

Abstract

This paper presents a novel approach to address pose-invariant face frontalization aiming Multiview Facial Expression Recognition (MFER). Particularly, the proposed approach is a hybrid method, including both synthesizing and mapping techniques. The key idea is to use mapped reconstructive coefficients of each arbitrary viewpoints and the frontal bases where the mapping functions are provided by learning between frontal and non-frontal faces’ coefficients. We also exploit sparse coding for synthesizing the frontalized faces, even with large poses. For evaluation, two qualitative and quantitative assessments are used along with an application of multiview facial expression recognition as a case study. The results show that our approach is efficient in terms of frontalizing non-frontal faces. Moreover, its validation on two popular datasets, BU3DFE and Multi-PIE, with various assessments contexts reveals its efficiency and stability on head pose variation, especially on large poses.