Winter Road Surface Conditions Classification using Convolutional Neural Network (CNN): Visible Light and Thermal Images Fusion

Abstract

In winter, road conditions play a crucial role in traffic flow efficiency and road safety. Icy, snowy, slushy, or wet road conditions reduce tire friction and affect vehicle stability which could lead to dangerous crashes. To keep traffic operations safe, cities spend a significant budget on winter maintenance operations such as snow plowing and spreading salt/sand. This paper proposes a methodology for automated winter road surface conditions classification using Convolutional Neural Network and the combination of thermal and visible light cameras. As part of this research, 4,244 pairs of visible light and thermal images are captured from pavement surfaces and classified into snowy, icy, wet, and slushy surface conditions. Two single-stream CNN models (visible light and thermal streams), and one dual-stream CNN model are developed. The average F1-Score of dual-stream model is 0.866, 0.935, 0.985, and 0.888 on snowy, icy, wet, and slushy, respectively. The weighted average F1-Score is 0.94.