Author:
Yang Zekun,Nakashima Yuta,Takemura Haruo
Abstract
AbstractHumor can be induced by various signals in the visual, linguistic, and vocal modalities emitted by humans. Finding humor in videos is an interesting but challenging task for an intelligent system. Previous methods predict humor in the sentence level given some text (e.g., speech transcript), sometimes together with other modalities, such as videos and speech. Such methods ignore humor caused by the visual modality in their design, since their prediction is made for a sentence. In this work, we first give new annotations to humor based on a sitcom by setting up temporal segments of ground truth humor derived from the laughter track. Then, we propose a method to find these temporal segments of humor. We adopt an approach based on sliding window, where the visual modality is described by pose and facial features along with the linguistic modality given as subtitles in each sliding window. We use long short-term memory networks to encode the temporal dependency in poses and facial features and pre-trained BERT to handle subtitles. Experimental results show that our method improves the performance of humor prediction.
Funder
China Scholarship Council
Japan Society for the Promotion of Science
Publisher
Springer Science and Business Media LLC
Subject
Computer Networks and Communications,Hardware and Architecture,Media Technology,Information Systems,Software
Reference29 articles.
1. Meyer, J.C.: Humor as a double-edged sword: four functions of humor in communication. Commun. Theory 10(3), 310–331 (2000)
2. Niculescu, A., van Dijk, B., Nijholt, A., Li, H., See, S.L.: Making social robots more attractive: the effects of voice pitch, humor and empathy. Int. J. Soc. Robot. 5, 171–191 (2013)
3. Mirnig, N., Stadler, S., Stollnberger, G., Giuliani, M., Tscheligi, M.: Robot humor: how self-irony and schadenfreude influence people’s rating of robot likability. In: 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), pp. 166–171 (2016)
4. Gray, C., Webster, T., Ozarowicz, B., Chen, Y., Bui, T., Srivastava, A., Fitter, N.T.: This bot knows what i’m talking about” humaninspired laughter classification methods for adaptive robotic comedians. In: 2022 31st IEEE International Conference on Robot and Human Interactive Communication (RO-MAN), pp. 1007–1014 (2022)
5. Kolb, W., Miller, T.: Human–computer Interaction in Pun Translation. Using Technologies for Creative-text Translation. Taylor & Francis, London (2022)
Cited by
1 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献