On the Application of Time Frequency Convolutional Neural Networks to Road Anomalies’ Identification with Accelerometers and Gyroscopes

Abstract

The detection and identification of road anomalies and obstacles in the road infrastructure has been investigated by the research community using different types of sensors. This paper evaluates the detection and identification of road anomalies/obstacles using the data collected from the Inertial Measurement Unit (IMU) installed in a vehicle and in particular from the data generated by the accelerometers’ and gyroscopes’ components. Inspired by the successes of the application of deep learning to various identification problems, this paper investigates the application of Convolutional Neural Network (CNN) to this specific problem. In particular, we propose a novel approach in this context where the time-frequency representation (i.e., spectrogram) is used as an input to the CNN rather than the original time domain data. This approach is evaluated on an experimental dataset collected using 12 different vehicles driving for more than 40 km of road. The results show that the proposed approach outperforms significantly and across different sampling rates both the application of CNN to the original time domain representation and the application of shallow machine learning algorithms. The approach achieves an identification accuracy of 97.2%. The results presented in this paper are based on an extensive optimization both of the CNN algorithm and the spectrogram implementation in terms of window size, type of window, and overlapping ratio. The accurate detection of road anomalies/obstacles could be useful to road infrastructure managers to monitor the quality of the road surface and to improve the accurate positioning of autonomous vehicles because road anomalies/obstacles could be used as landmarks.