Improving the accuracy of object classification on the ground based on a combination of sensory modalities using binary cross-tropism as a loss function.

Abstract

In today's warfare environment, software based on machine learning techniques for detecting and classifying infrastructure objects on the ground has become extremely important. In this regard, the task of improving the accuracy of classification of objects on the ground is becoming more urgent, as the use of UAVs and space systems is vital for intelligence activities. Given the nature of the input data, namely static terrain images obtained in the form of satellite images and UAV camera images, it is advisable to use convolutional neural networks to solve classification problems. In most cases, satellite images are presented in the form of multispectral and hyperspectral images, so publicly available datasets offered by the SpaceNet research community were used to train the model. An important step in preparing the training set is image orthorectification, namely adding 3D surface information to the images, which provides the model with important geometric information for semantic classes such as buildings and other structures, corrects geometric distortions, and helps the model to recognize objects in a consistent geospatial context. In the course of the experiments, the SegNet model was trained with and without the normalized Digital Surface Model (nDSM). The experimental results show that the generalized classification accuracy for six classes of objects on the test dataset increases by 23.9%. And experiments with training set limitation demonstrate that, if necessary, it is enough to use half of the available training data set to obtain only 4% lower classification accuracy and save about 10 hours of training.