Quantitative and visual analysis of the impact of music on perceived emotion of film

Abstract

This article presents quantitative and visual methods for the analysis of the effect of music on emotion perceived in film. We discover strong, visible, and quantifiable trends in the effect of music on perceived emotion of film. We perform studies using both selected classical and composed music segments annotated with diverse emotions, paired with ambiguous film clips. We collect and analyze viewers' ratings of stress, activity, and dominance in the silent film, and film with various music soundtracks. The results are mapped onto a three-dimensional emotion space for visual and quantitative analysis. Aggregate scatter plots and center of mass results are presented using this emotion space. We find that the center of mass of the perceived emotion of film and music combined is consistently situated on a path between the center of mass of the musicalone response and that of the film-alone response. Regression analysis based on this trajectory observation results in coefficients with high R 2 values (R 2 = 0.675 for stress, R 2 = 0.817 for activity, and R 2 = 0.813 for dominance) for the first study with classical music selections, and lower R 2 values (R 2 = 0.199 for stress, R 2 = 0.405 for activity, and R 2 = 0.660 for dominance) for the second study with composed music. We conclude that continuous treatment of the two-dimensional emotion space provides a metric for comparing and assessing emotion ratings, and that spatial interpolation in this space provides a viable method for predicting the effect of music on perceived emotion of film.