Toward Unified and Quantitative Cinematic Shot Attribute Analysis
-
Published:2023-10-08
Issue:19
Volume:12
Page:4174
-
ISSN:2079-9292
-
Container-title:Electronics
-
language:en
-
Short-container-title:Electronics
Author:
Li Yuzhi1ORCID, Tian Feng1, Xu Haojun1, Lu Tianfeng1
Affiliation:
1. Shanghai Film Academy, Shanghai University, Shanghai 200073, China
Abstract
Cinematic Shot Attribute Analysis aims to analyze the intrinsic attributes of movie shots, such as movement and scale. In previous methods, specialized architectures were designed for each specific task and relied on the use of optical flow maps. In this paper, we consider shot attribute analysis as a unified task of motion–static weight allocation, and propose a motion–static dual-path architecture for recognizing various shot attributes. In this architecture, we design a new action cue generation module for adapting the end-to-end training process instead of a pre-trained optical flow network; and, to address the issue of limited samples in movie shot datasets, we design a fixed-size adjustment strategy to enable the network to directly utilize pre-trained vision transformer models while adapting to shot data inputs at arbitrary sample rates. In addition, we quantitatively analyze the sensitivity of different shot attributes to motion and static features for the first time. Subsequent experimental results on two datasets, MovieShots and AVE, demonstrate that our proposed method outperforms all previous approaches without increasing computational cost.
Subject
Electrical and Electronic Engineering,Computer Networks and Communications,Hardware and Architecture,Signal Processing,Control and Systems Engineering
Reference45 articles.
1. Rao, A., Wang, J., Xu, L., Jiang, X., Huang, Q., Zhou, B., and Lin, D. (2020, January 23–28). A unified framework for shot type classification based on subject centric lens. Proceedings of the Computer Vision–ECCV 2020: 16th European Conference, Glasgow, UK. Part XI 16. 2. Souček, T., and Lokoč, J. (2020). Transnet v2: An effective deep network architecture for fast shot transition detection. arXiv. 3. Rao, A., Xu, L., Xiong, Y., Xu, G., Huang, Q., Zhou, B., and Lin, D. (2020, January 13–19). A local-to-global approach to multi-modal movie scene segmentation. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA. 4. Carreira, J., and Zisserman, A. (2017, January 21–26). Quo vadis, action recognition? A new model and the kinetics dataset. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, HI, USA. 5. Learning a discriminative mid-level feature for action recognition;Liu;Sci. China Inf. Sci.,2014
|
|