A LiDAR-less approach to autonomous hazard detection and avoidance systems based on semantic segmentation

Abstract

AbstractIn this paper, a passive hazard detection and avoidance (HDA) system is presented, relying only on images as observations. To process these images, convolutional neural networks (CNNs) are used to perform semantic segmentation and identify hazards corresponding to three different layers, namely feature detection, shadow detection, and slope estimation. The absence of active sensors such as light detection and ranging (LiDAR) makes it challenging to assess the surface geometry of a celestial body, and the training of the neural networks in this work is oriented towards coping with that drawback. The image data set for the training is generated using blender, and different body shape models (also referred to as meshes) are included, onto which stochastic feature populations and illumination conditions are imposed to produce a more diverse database. The CNNs are trained following a transfer learning approach to reduce the training effort and take advantage of previously trained networks. The results accurately predict the hazards in the images belonging to the data set, but struggle to yield successful predictions for the slope estimation, when images external to the data set are used, indicating that including the geometry of the target body in the training phase makes an impact on the quality of these predictions. The obtained predictions are composed to create safety maps, which are meant to be given as input to the guidance block of the spacecraft to evaluate the need for a manoeuvre to avoid hazardous areas. Additionally, preliminary hardware-in-the-loop (HIL) test results are included, in which the algorithms developed are confronted against images taken using real hardware.