Asymmetric-Convolution-Guided Multipath Fusion for Real-Time Semantic Segmentation Networks

Abstract

Aiming to provide solutions for problems proposed by the inaccurate segmentation of long objects and information loss of small objects in real-time semantic segmentation algorithms, this paper proposes a lightweight multi-branch real-time semantic segmentation network based on BiseNetV2. The new auxiliary branch makes full use of spatial details and context information to cover the long object in the field of view. Meanwhile, in order to ensure the inference speed of the model, the asymmetric convolution is used in each stage of the auxiliary branch to design a structure with low computational complexity. In the multi-branch fusion stage, the alignment-and-fusion module is designed to provide guidance information for deep and shallow feature mapping, so as to make up for the problem of feature misalignment in the fusion of information at different scales, and thus reduce the loss of small target information. In order to further improve the model’s awareness of key information, a global context module is designed to capture the most important features in the input data. The proposed network uses an NVIDIA GeForce RTX 3080 Laptop GPU experiment on the road street view Cityscapes and CamVid datasets, with the average simultaneously occurring ratios reaching 77.1% and 77.4%, respectively, and the running speeds reaching 127 frames/s and 112 frames/s, respectively. The experimental results show that the proposed algorithm can achieve a real-time segmentation and improve the accuracy significantly, showing good semantic segmentation performance.