LLNet: A Fusion Classification Network for Land Localization in Real-World Scenarios

Abstract

Lane localization is one of the core tasks in an autonomous driving system. It receives the visual information collected by the camera and the lane marks and road edges information outputted from the perception module and gives lane index for the subsequent decision module. Traditional rule-based lane localization methods using navigation maps can only be effective in regular road scenarios and have poor generalization ability. High-Definition Map (HD map) was originally thought to solve the lane localization problem, but due to the regulations of the relevant departments, HD map is currently not allowed to be used in autonomous driving systems. In addition, many multi-sensor fusion methods have been proposed to solve the lane localization problem. However, due to the extremely strict safety requirements of autonomous driving systems, these well-designed solutions make it difficult to meet the requirements in terms of robustness, efficiency, and stability. To solve these problems, we innovatively define the lane localization task as a classification problem. First, to better utilize the perceptual information outputted from the perceptual model, we design an image-generating method that projects the perceptual information onto a new image and ensures that our model can learn the perceptual features wisely. Second, to better fuse the perceptual and visual information, we propose a fusion structure deep learning neural network named LLNet to address the lane localization problem in an end-to-end manner. Finally, to ensure the generalization ability, robustness, and stability of LLNet, we conduct extensive comparison experiments on a large-scale real-world dataset, with a total mileage of over 400 km. The experiments show that our approach remarkably outperforms the deep learning classification baselines. In the discussion part of this paper, we give a comprehensive and detailed elaboration for the effectiveness of various designs in our LLNet. To our knowledge, LLNet is the first lane localization method based entirely on deep learning. LLNet is added to the self-driving suite for a kind of mass production vehicle that will be available in the summer of 2022, with an expected sales volume more than 300,000.