Author:
Chen Jingyuan,Ma Lin,Chen Xinpeng,Jie Zequn,Luo Jiebo
Abstract
In this paper, we consider the task of natural language video localization (NLVL): given an untrimmed video and a natural language description, the goal is to localize a segment in the video which semantically corresponds to the given natural language description. We propose a localizing network (LNet), working in an end-to-end fashion, to tackle the NLVL task. We first match the natural sentence and video sequence by cross-gated attended recurrent networks to exploit their fine-grained interactions and generate a sentence-aware video representation. A self interactor is proposed to perform crossframe matching, which dynamically encodes and aggregates the matching evidences. Finally, a boundary model is proposed to locate the positions of video segments corresponding to the natural sentence description by predicting the starting and ending points of the segment. Extensive experiments conducted on the public TACoS and DiDeMo datasets demonstrate that our proposed model performs effectively and efficiently against the state-of-the-art approaches.
Publisher
Association for the Advancement of Artificial Intelligence (AAAI)
Cited by
21 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
1. Efficient Spatio-Temporal Video Grounding with Semantic-Guided Feature Decomposition;Proceedings of the 31st ACM International Conference on Multimedia;2023-10-26
2. Temporal Sentence Grounding in Videos: A Survey and Future Directions;IEEE Transactions on Pattern Analysis and Machine Intelligence;2023-08
3. A Survey on Video Moment Localization;ACM Computing Surveys;2023-01-16
4. Cross-Graph Transformer Network for Temporal Sentence Grounding;Artificial Neural Networks and Machine Learning – ICANN 2023;2023
5. Gaussian Kernel-Based Cross Modal Network for Spatio-Temporal Video Grounding;2022 IEEE International Conference on Image Processing (ICIP);2022-10-16