Multimodal Analysis and Prediction of Persuasiveness in Online Social Multimedia

Abstract

Our lives are heavily influenced by persuasive communication, and it is essential in almost any type of social interaction from business negotiation to conversation with our friends and family. With the rapid growth of social multimedia websites, it is becoming ever more important and useful to understand persuasiveness in the context of social multimedia content online. In this article, we introduce a newly created multimedia corpus of 1,000 movie review videos with subjective annotations of persuasiveness and related high-level characteristics or attributes (e.g., confidence). This dataset will be made freely available to the research community. We designed our experiments around the following five main research hypotheses. First, we study if computational descriptors derived from verbal and nonverbal behavior can be predictive of persuasiveness. We further explore combining descriptors from multiple communication modalities (acoustic, verbal, para-verbal, and visual) for predicting persuasiveness and compare with using a single modality alone. Second, we investigate how certain high-level attributes, such as credibility or expertise, are related to persuasiveness and how the information can be used in modeling and predicting persuasiveness. Third, we investigate differences when speakers are expressing a positive or negative opinion and if the opinion polarity has any influence in the persuasiveness prediction. Fourth, we further study if gender has any influence in the prediction performance. Last, we test if it is possible to make comparable predictions of persuasiveness by only looking at thin slices (i.e., shorter time windows) of a speaker's behavior.