Facial Makeup Detection Using Multi-Scale Local Binary Patterns and Convolutional Neural Network Fusion

Abstract

This study presents a novel facial makeup detection scheme using fusion of multi-scale Local Binary Patterns (ms-LBP) texture methods and convolutional neural network (CNN) deep learning extractor. Facial makeups affect the accuracy of face recognition systems due to appearance alteration of individuals. In order to fuse the facial features, the proposed scheme first considers concatenation of extracted global histogram of a whole image using three different LBP operators. The LBP scales are used to extract and then concatenate the local histogram of overlapping and nonoverlapping blocks of images. This way utilizes the advantages of microtexton information of local primitives besides the extracted global textures. Finally, the extracted features using CNN feature extractor are combined with global and local multi-scale textures. In general, CNN deep learning extractor learns high-level discriminative characteristics of images and therefore the proposed system improves the facial makeup detection rate by involving both microtexton and discriminative information of facial images. In addition, the proposed method attempts to select the optimized subset of facial features to increase the detection performance of system by applying Particle Swarm Optimization (PSO) technique after feature level fusion. The paper uses Support Vector Machine (SVM) classifier for classifying the facial vectors into makeup or no-makeup classes. The proposed scheme is then evaluated using YMU, VMU and MIW facial makeup databases with consideration of light, medium and heavy makeups on several datasets. Experimental results analysis clarifies the effectiveness of proposed facial makeup detection framework of this study.