End-to-End Detection of a Landing Platform for Offshore UAVs Based on a Multimodal Early Fusion Approach
Author:
Neves Francisco Soares12ORCID, Claro Rafael Marques12ORCID, Pinto Andry Maykol12ORCID
Affiliation:
1. Faculty of Engineering, University of Porto (FEUP), 4200-465 Porto, Portugal 2. Centre for Robotics and Autonomous Systems—INESC TEC, 4200-465 Porto, Portugal
Abstract
A perception module is a vital component of a modern robotic system. Vision, radar, thermal, and LiDAR are the most common choices of sensors for environmental awareness. Relying on singular sources of information is prone to be affected by specific environmental conditions (e.g., visual cameras are affected by glary or dark environments). Thus, relying on different sensors is an essential step to introduce robustness against various environmental conditions. Hence, a perception system with sensor fusion capabilities produces the desired redundant and reliable awareness critical for real-world systems. This paper proposes a novel early fusion module that is reliable against individual cases of sensor failure when detecting an offshore maritime platform for UAV landing. The model explores the early fusion of a still unexplored combination of visual, infrared, and LiDAR modalities. The contribution is described by suggesting a simple methodology that intends to facilitate the training and inference of a lightweight state-of-the-art object detector. The early fusion based detector achieves solid detection recalls up to 99% for all cases of sensor failure and extreme weather conditions such as glary, dark, and foggy scenarios in fair real-time inference duration below 6 ms.
Funder
European Union FLY.PT-P2020 Mobilizado
Subject
Electrical and Electronic Engineering,Biochemistry,Instrumentation,Atomic and Molecular Physics, and Optics,Analytical Chemistry
Reference41 articles.
1. Pendleton, S.D., Andersen, H., Du, X., Shen, X., Meghjani, M., Eng, Y.H., Rus, D., and Ang, M.H. (2017). Perception, planning, control, and coordination for autonomous vehicles. Machines, 5. 2. Lim, T.Y., Ansari, A., Major, B., Fontijne, D., Hamilton, M., Gowaikar, R., and Subramanian, S. (2019, January 14). Radar and camera early fusion for vehicle detection in advanced driver assistance systems. Proceedings of the Machine Learning for Autonomous Driving Workshop at the 33rd Conference on Neural Information Processing Systems, Vancouver, BC, Canada. 3. Yeong, D.J., Velasco-Hernandez, G., Barry, J., and Walsh, J. (2021). Sensor and Sensor Fusion Technology in Autonomous Vehicles: A Review. Sensors, 21. 4. Kim, J., Koh, J., Kim, Y., Choi, J., Hwang, Y., and Choi, J.W. (2018, January 2–6). Robust deep multi-modal learning based on gated information fusion network. Proceedings of the Asian Conference on Computer Vision, Perth, Australia. 5. Early, intermediate and late fusion strategies for robust deep learning-based multimodal action recognition;Boulahia;Mach. Vis. Appl.,2021
Cited by
5 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
|
|