Multi-Modal Scene Duplicate Detection from News Videos Focusing on Human Faces

Abstract

In this paper, as a tool for structuring a large volume of archived news videos according to their semantic contents, we propose a method that effectively detects scene duplicates, assuming the presence of a person speaking in the videos. A scene duplicate is defined here as a pair of video segments taken at the same event from different viewpoints. When considering scenes where a subject is speaking in news videos, referring to the audio channel could be effective to detect scene duplicates regardless of viewpoints. However, it cannot be relied on when external audio sources overlap the original one or when the subject is actually not speaking. In contrast, the image channel can be useful in most cases. However, significant difference in viewpoints could prevent accurate detection. Therefore, we propose a method that combines the results obtained from both audio and image channels in order to improve the accuracy of scene duplicate detection from news videos. The proposed method was evaluated through an experiment with actual broadcast news videos by comparing it with a conventional method. As a result, we confirmed that the detection accuracy significantly improved by the proposed method in both recall and precision.