Boosting Semantic Segmentation by Conditioning the Backbone with Semantic Boundaries
Author:
Ishikawa Haruya1ORCID, Aoki Yoshimitsu1ORCID
Affiliation:
1. Department of Electronics and Electrical Engineering, Facility of Science and Technology, Keio University, 3-14-1, Hiyoshi, Kohoku-ku, Yokohama 223-8522, Japan
Abstract
In this paper, we propose the Semantic-Boundary-Conditioned Backbone (SBCB) framework, an effective approach to enhancing semantic segmentation performance, particularly around mask boundaries, while maintaining compatibility with various segmentation architectures. Our objective is to improve existing models by leveraging semantic boundary information as an auxiliary task. The SBCB framework incorporates a complementary semantic boundary detection (SBD) task with a multi-task learning approach. It enhances the segmentation backbone without introducing additional parameters during inference or relying on independent post-processing modules. The SBD head utilizes multi-scale features from the backbone, learning low-level features in early stages and understanding high-level semantics in later stages. This complements common semantic segmentation architectures, where features from later stages are used for classification. Extensive evaluations using popular segmentation heads and backbones demonstrate the effectiveness of the SBCB. It leads to an average improvement of 1.2% in IoU and a 2.6% gain in the boundary F-score on the Cityscapes dataset. The SBCB framework also improves over- and under-segmentation characteristics. Furthermore, the SBCB adapts well to customized backbones and emerging vision transformer models, consistently achieving superior performance. In summary, the SBCB framework significantly boosts segmentation performance, especially around boundaries, without introducing complexity to the models. Leveraging the SBD task as an auxiliary objective, our approach demonstrates consistent improvements on various benchmarks, confirming its potential for advancing the field of semantic segmentation.
Funder
Japan Society for the Promotion of Science
Subject
Electrical and Electronic Engineering,Biochemistry,Instrumentation,Atomic and Molecular Physics, and Optics,Analytical Chemistry
Reference69 articles.
1. Cheng, B., Girshick, R.B., Doll’ar, P., Berg, A.C., and Kirillov, A. (2021, January 19–25). Boundary IoU: Improving Object-Centric Image Segmentation Evaluation. Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Virtual. 2. Takikawa, T., Acuna, D., Jampani, V., and Fidler, S. (November, January 27). Gated-SCNN: Gated Shape CNNs for Semantic Segmentation. Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision (ICCV), Seoul, Republic of Korea. 3. Li, X., Li, X., Zhang, L., Cheng, G., Shi, J., Lin, Z., Tan, S., and Tong, Y. (2020, January 23–28). Improving Semantic Segmentation via Decoupled Body and Edge Supervision. Proceedings of the 2020 IEEE/CVF European Conference on Computer Vision (ECCV), Virtual. 4. Zhen, M., Wang, J., Zhou, L., Li, S., Shen, T., Shang, J., Fang, T., and Long, Q. (2020, January 13–19). Joint Semantic Segmentation and Boundary Detection Using Iterative Pyramid Contexts. Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Virtual. 5. Yu, Z., Huang, R., Byeon, W., Liu, S., Liu, G., Breuel, T., Anandkumar, A., and Kautz, J. (2021, January 6–14). Coupled Segmentation and Edge Learning via Dynamic Graph Propagation. Proceedings of the Advances in Neural Information Processing Systems (NeurIPS), Virtual.
Cited by
1 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
1. Implementing ViT Models for Traffic Sign Detection in Autonomous Driving Systems;2024 5th International Conference on Recent Trends in Computer Science and Technology (ICRTCST);2024-04-09
|
|