Conscious observational behavior in recognizing landmarks in facial expressions

Abstract

The present study investigated (1) how well humans can recognize facial expressions represented by a small set of landmarks, a commonly used technique in facial recognition in machine learning and (2) differences in conscious observational behaviors to recognized different types of expressions. Our video stimuli consisted of facial expression represented by 68 landmark points. Conscious observational behaviors were measured by movements of the mouse cursor where a small area around it was only visible to participants. We constructed Bayesian models to analyze how personality traits and observational behaviors influenced how participants recognized different facial expressions. We found that humans could recognize positive expressions with high accuracy, similar to machine learning, even when faces were represented by a small set of landmarks. Although humans fared better than machine learning, recognition of negative expressions was not as high as positives. Our results also showed that personality traits and conscious observational behaviors significantly influenced recognizing facial expressions. For example, people with high agreeableness could correctly recognize faces expressing happiness by observing several areas among faces without focusing on any specific part for very long. These results suggest a mechanism whereby personality traits lead to different conscious observational behaviors and recognitions of facial expressions are based on information obtained through those observational behaviors.