A Thermal Blended Facial Expression Analysis and Recognition System Using Deformed Thermal Facial Areas

Abstract

There are many research works in visible as well as thermal facial expression analysis and recognition. Several facial expression databases have been designed in both modalities. However, little attention has been given for analyzing blended facial expressions in the thermal infrared spectrum. In this paper, we have introduced a Visual-Thermal Blended Facial Expression Database (VTBE) that contains visual and thermal face images with both basic and blended facial expressions. The database contains 12 posed blended facial expressions and spontaneous six basic facial expressions in both modalities. In this paper, we have proposed Deformed Thermal Facial Area (DTFA) in thermal expressive face image and make an analysis to differentiate between basic and blended expressions using DTFA. Here, the fusion of DTFA and Deformed Visual Facial Area (DVFA) has been proposed combining the features of both modalities and experiments and has been conducted on this new database. However, to show the effectiveness of our proposed approach, we have compared our method with state-of-the-art methods using USTC-NVIE database. Experiment results reveal that our approach is superior to state-of-the-art methods.