Facial expression recognition under occlusion conditions based on multi-feature cross-attention

Abstract

Although facial expression recognition (FER) has a wide range of applications, it may be difficult to achieve under local occlusion conditions which may result in the loss of valuable expression features. This issue has motivated the present study, as a part of which an effective multi-feature cross-attention network (MFCA-Net) is proposed. The MFCA-Net consists of a two-branch network comprising a multi-feature convolution module and a local cross-attention module. Thus, it enables decomposition of facial features into multiple sub-features by the multi-feature convolution module to reduce the impact of local occlusion on facial expression feature extraction. In the next step, the local cross-attention module distinguishes between occluded and unoccluded sub-features and focuses on the latter to facilitate FER. When the MFCA-Net performance is evaluated by applying it to three public large-scale datasets (RAF-DB, FERPlus, and AffectNet), the experimental results confirm its good robustness. Further validation is performed on a real FER dataset with local occlusion of the face.