DIPS: A Dyadic Impression Prediction System for Group Interaction Videos

Abstract

We consider the problem of predicting the impression that one subject has of another in a video clip showing a group of interacting people. Our novel Dyadic Impression Prediction System ( DIPS ) contains two major innovations. First, we develop a novel method to align the facial expressions of subjects p i and p j as well as account for the temporal delay that might be involved in p i reacting to p j ’s facial expressions. Second, we propose the concept of a multilayered stochastic network for impression prediction on top of which we build a novel Temporal Delayed Network graph neural network architecture. Our overall DIPS architecture predicts six dependent variables relating to the impression p i has of p j . Our experiments show that DIPS beats eight baselines from the literature, yielding statistically significant improvements of 19.9% to 30.8% in AUC and 12.6% to 47.2% in F1-score. We further conduct ablation studies showing that our novel features contribute to the overall quality of the predictions made by DIPS .