3D‐Facial Expression Synthesis and its Application to Face Recognition Systems

Abstract

One of the main problems in Face Recognition systems is the recognition of an input face with a different expression than the available in the training database. In this work, we propose a new 3D‐face expression synthesis approach for expression independent face recognition systems (FRS). Different than current schemes in the literature, all the steps involved in our approach (face denoising, registration, and expression synthesis) are performed in the 3D domain. Our final goal is to increase the flexibility of 3D‐FRS by allowing them to artificially generate multiple face expressions from a neutral expression face. A generic 3D‐range image is modeled by the Finite Element Method with three simplified layers representing the skin, fatty tissue and the cranium. The face muscular anatomy is superimposed to the 3D model for the synthesis of expressions. Our approach can be divided into three main steps: Denoising Algorithm, which is applied to remove long peaks present in the original 3Dface samples; Automatic Control Points Detection, to detect particular facial landmarks such as eye and mouth corners, nose tip, etc., helpful in the recognition process; Face Registration of a 3D‐face model with each sample face with neutral expression in the training database in order to augment its training set (with 18 predefined expressions). Additional expressions can be learned from input faces or an unknown expression can be transformed to the closest known expression. Our results show that the 3D‐face model resembles perfectly the neutral expression faces in the training database while providing a natural change of expression. Moreover, the inclusion of our expression synthesis approach in a simple 3D‐FRS based on Fisherfaces increased significantly the recognition rate without requiring complex 3D‐face recognition chemes.