MAFFNet: real-time multi-level attention feature fusion network with RGB-D semantic segmentation for autonomous driving

Abstract

Compared with RGB semantic segmentation, RGB-D semantic segmentation can combine the geometric depth information to effectively improve the segmentation accuracy. Considering the application of RGB-D semantic segmentation in autonomous driving, we design a real-time semantic segmentation network, that is, MAFFNet, which can effectively extract depth features and combine the complementary information in RGB and depth. We also design a multi-level attention feature fusion module that can excavate the available context information of RGB and depth features. At the same time, its inference speed can also meet the demands of autonomous driving. Experiments show that our network achieves excellent performance of 74.4% mIoU and an inference speed of 15.9 Hz at a full resolution of 2048 × 1024 on the cityscapes dataset. Using multi-source learning, we mixed the cityscapes and lost and found as the multi-dataset. Our network is also superior to previous algorithms in using the multi-dataset to detect small obstacles outside the road.