Recurrent Graph Neural Networks for Video Instance Segmentation
-
Published:2022-11-18
Issue:2
Volume:131
Page:471-495
-
ISSN:0920-5691
-
Container-title:International Journal of Computer Vision
-
language:en
-
Short-container-title:Int J Comput Vis
Author:
Brissman Emil, Johnander JoakimORCID, Danelljan Martin, Felsberg Michael
Abstract
AbstractVideo instance segmentation is one of the core problems in computer vision. Formulating a purely learning-based method, which models the generic track management required to solve the video instance segmentation task, is a highly challenging problem. In this work, we propose a novel learning framework where the entire video instance segmentation problem is modeled jointly. To this end, we design a graph neural network that in each frame jointly processes all detections and a memory of previously seen tracks. Past information is considered and processed via a recurrent connection. We demonstrate the effectiveness of the proposed approach in comprehensive experiments. Our approach operates online at over 25 FPS and obtains 16.3 AP on the challenging OVIS benchmark, setting a new state-of-the-art. We further conduct detailed ablative experiments that validate the different aspects of our approach. Code is available at https://github.com/emibr948/RGNNVIS-PlusPlus.
Funder
Knut och Alice Wallenbergs Stiftelse
Publisher
Springer Science and Business Media LLC
Subject
Artificial Intelligence,Computer Vision and Pattern Recognition,Software
Reference50 articles.
1. Athar, A., Mahadevan, S., Ošep, A., Leal-Taixé, L., & Leibe, B. (2020). Stem-seg: Spatio-temporal embeddings for instance segmentation in videos. In Computer Vision - ECCV 2020–16th European Conference, Glasgow, UK, August 23–28, 2020. Proceedings, Part XI, Lecture Notes in Computer Science, 12356, 158–177. 2. Athar, A., Mahadevan, S., Ošep, A., Leal-Taixé, L., & Leibe, B. (2021). A single-stage, bottom-up approach for occluded vis using spatio-temporal embeddings. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, pp. 3858–3862. 3. Ba, J.L., Kiros, J.R., & Hinton, G.E. (2016). Layer normalization 4. Battaglia, P.W., Hamrick, J.B., Bapst, V., Sanchez-Gonzalez, A., Zambaldi, V.F., Malinowski, M., Tacchetti, A., Raposo, D., Santoro, A., Faulkner, R., Gülçehre, Ç., Song, H.F., Ballard, A.J., Gilmer, J., Dahl, G.E., Vaswani, A., Allen, K.R., Nash, C., Langston, V., Dyer, C., Heess, N., Wierstra, D., Kohli, P., Botvinick, M., Vinyals, O., Li, Y., Pascanu, R. (2018). Relational inductive biases, deep learning, and graph networks. CoRR arXiv:1806.01261. 5. Benenson, R., Popov, S., & Ferrari, V. (2019). Large-scale interactive object segmentation with human annotators. In CVPR.
Cited by
5 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
|
|