Combining Facial Parts For Learning Gender, Ethnicity, and Emotional State Based on RGB-D Information

Abstract

With the success of emerging RGB-D cameras such as the Kinect sensor, combining the shape (depth) and texture information to improve the quality of recognition became a trend among computer vision researchers. In this work, we address the problem of face classification in the context of RGB images and depth data. Inspired by the psychological results for human face perception, this article focuses on (i) finding out which facial parts are most effective at making the difference for some social aspects of face perception (gender, ethnicity, and emotional state), (ii) determining the optimal decision by combining the decision rendered by the individual parts, and (iii) extracting the promising features from RGB-D faces to exploit all the potential that this data provide. Experimental results on EurecomKinect Face and CurtinFaces databases show that the proposed approach improves the recognition quality in many use cases.