Advanced techniques for automated emotion recognition in dogs from video data through deep learning

Abstract

AbstractInter-species emotional relationships, particularly the symbiotic interaction between humans and dogs, are complex and intriguing. Humans and dogs share fundamental mammalian neural mechanisms including mirror neurons, crucial to empathy and social behavior. Mirror neurons are activated during the execution and observation of actions, indicating inherent connections in social dynamics across species despite variations in emotional expression. This study explores the feasibility of using deep-learning Artificial Intelligence systems to accurately recognize canine emotions in general environments, to assist individuals without specialized knowledge or skills in discerning dog behavior, particularly related to aggression or friendliness. Starting with identifying key challenges in classifying pleasant and unpleasant emotions in dogs, we tested advanced deep-learning techniques and aggregated results to distinguish potentially dangerous human--dog interactions. Knowledge transfer is used to fine-tune different networks, and results are compared on original and transformed sets of frames from the Dog Clips dataset to investigate whether DogFACS action codes detailing relevant dog movements can aid the emotion recognition task. Elaborating on challenges and biases, we emphasize the need for bias mitigation to optimize performance, including different image preprocessing strategies for noise mitigation in dog recognition (i.e., face bounding boxes, segmentation of the face or body, isolating the dog on a white background, blurring the original background). Systematic experimental results demonstrate the system’s capability to accurately detect emotions and effectively identify dangerous situations or signs of discomfort in the presence of humans.