Enhancing Semi Supervised Semantic Segmentation Through Cycle-Consistent Label Propagation in Video

Abstract

AbstractTo perform semantic image segmentation using deep learning models, a significant quantity of data and meticulous manual annotation is necessary (Mani in: Research anthology on improving medical imaging techniques for analysis and intervention. IGI Global, pp. 107–125, 2023), and the process consumes a lot of resources, including time and money. To resolve such issues, we introduce a unique label propagation method (Qin et al. in IEEE/CAA J Autom Sinica 10(5):1192–1208, 2023) that utilizes cycle consistency across time to propagate labels over longer time horizons with higher accuracy. Additionally, we acknowledge that dense pixel annotation is a noisy process (Das et al. in: Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp. 5978–5987, 2023), whether performed manually or automatically. To address this, we present a principled approach that accounts for label uncertainty when training with labels from multiple noisy labeling processes. We introduce two new approaches; Warp-Refine Propagation and Uncertainty-Aware Training, for improving label propagation and handling noisy labels, respectively, and support the process with quantitative and qualitative evaluations and theoretical justification. Our contributions are validated on the Cityscapes and ApolloScape datasets, where we achieve encouraging results. In later endeavors, the aim should be to expand such approaches to include other noisy augmentation processes like image-based rendering methods (Laraqui et al. in Int J Comput Aid Eng Technol 18(5):141–151, 2023), thanks to the noisy label learning approach.