Enhanced Perception for Autonomous Driving Using Semantic and Geometric Data Fusion

Author:

Florea HoratiuORCID,Petrovai AndraORCID,Giosan IonORCID,Oniga FlorinORCID,Varga RobertORCID,Nedevschi SergiuORCID

Abstract

Environment perception remains one of the key tasks in autonomous driving for which solutions have yet to reach maturity. Multi-modal approaches benefit from the complementary physical properties specific to each sensor technology used, boosting overall performance. The added complexity brought on by data fusion processes is not trivial to solve, with design decisions heavily influencing the balance between quality and latency of the results. In this paper we present our novel real-time, 360∘ enhanced perception component based on low-level fusion between geometry provided by the LiDAR-based 3D point clouds and semantic scene information obtained from multiple RGB cameras, of multiple types. This multi-modal, multi-sensor scheme enables better range coverage, improved detection and classification quality with increased robustness. Semantic, instance and panoptic segmentations of 2D data are computed using efficient deep-learning-based algorithms, while 3D point clouds are segmented using a fast, traditional voxel-based solution. Finally, the fusion obtained through point-to-image projection yields a semantically enhanced 3D point cloud that allows enhanced perception through 3D detection refinement and 3D object classification. The planning and control systems of the vehicle receives the individual sensors’ perception together with the enhanced one, as well as the semantically enhanced 3D points. The developed perception solutions are successfully integrated onto an autonomous vehicle software stack, as part of the UP-Drive project.

Funder

European Union

Ministerul Cercetării și Inovării

Publisher

MDPI AG

Subject

Electrical and Electronic Engineering,Biochemistry,Instrumentation,Atomic and Molecular Physics, and Optics,Analytical Chemistry

Reference54 articles.

1. UP-Drive H2020 European Union Program—Grant 688652 https://up-drive.ethz.ch/

2. Map Management for Efficient Long-Term Visual Localization in Outdoor Environments;Buerki;Proceedings of the IEEE Intelligent Vehicles Symposium (IV),2018

3. Super-sensor for 360-degree environment perception: Point cloud segmentation using image features;Varga;Proceedings of the 2017 IEEE 20th International Conference on Intelligent Transportation Systems (ITSC),2017

4. EB Assist ADTF-Automotive Data and Time-Triggered Framework https://www.elektrobit.com/products/automated-driving/eb-assist/adtf/

5. Are we ready for autonomous driving? the kitti vision benchmark suite;Geiger;Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition,2012

Cited by 16 articles. 订阅此论文施引文献 订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献

同舟云学术

1.学者识别学者识别

2.学术分析学术分析

3.人才评估人才评估

"同舟云学术"是以全球学者为主线,采集、加工和组织学术论文而形成的新型学术文献查询和分析系统,可以对全球学者进行文献检索和人才价值评估。用户可以通过关注某些学科领域的顶尖人物而持续追踪该领域的学科进展和研究前沿。经过近期的数据扩容,当前同舟云学术共收录了国内外主流学术期刊6万余种,收集的期刊论文及会议论文总量共计约1.5亿篇,并以每天添加12000余篇中外论文的速度递增。我们也可以为用户提供个性化、定制化的学者数据。欢迎来电咨询!咨询电话:010-8811{复制后删除}0370

www.globalauthorid.com

TOP

Copyright © 2019-2024 北京同舟云网络信息技术有限公司
京公网安备11010802033243号  京ICP备18003416号-3