Author:
Yu Xiaodong,Shi Dahu,Wei Xing,Ren Ye,Ye Tingqun,Tan Wenming
Abstract
This paper presents an end-to-end instance segmentation framework, termed SOIT, that Segments Objects with Instance-aware Transformers. Inspired by DETR, our method views instance segmentation as a direct set prediction problem and effectively removes the need for many hand-crafted components like RoI cropping, one-to-many label assignment, and non-maximum suppression (NMS). In SOIT, multiple queries are learned to directly reason a set of object embeddings of semantic category, bounding-box location, and pixel-wise mask in parallel under the global image context. The class and bounding-box can be easily embedded by a fixed-length vector. The pixel-wise mask, especially, is embedded by a group of parameters to construct a lightweight instance-aware transformer. Afterward, a full-resolution mask is produced by the instance-aware transformer without involving any RoI-based operation. Overall, SOIT introduces a simple single-stage instance segmentation framework that is both RoI- and NMS-free. Experimental results on the MS COCO dataset demonstrate that SOIT outperforms state-of-the-art instance segmentation approaches significantly. Moreover, the joint learning of multiple tasks in a unified query embedding can also substantially improve the detection performance. Code is available at https://github.com/yuxiaodongHRI/SOIT.
Publisher
Association for the Advancement of Artificial Intelligence (AAAI)
Cited by
9 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
1. ReSParser: Fully Convolutional Multiple Human Parsing With Representative Sets;IEEE Transactions on Multimedia;2024
2. DQFORMER: Dynamic Query Transformer for Lane Detection;ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP);2023-06-04
3. Autoregressive Visual Tracking;2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR);2023-06
4. CircleFormer: Circular Nuclei Detection in Whole Slide Images with Circle Queries and Attention;Lecture Notes in Computer Science;2023
5. Vision Transformer-based recognition tasks: a critical review;Journal of Image and Graphics;2023