Real-time automated deep learning based railroad trespassing violation detection and tracking at highway-rail grade crossing

Abstract

Abstract Trespassing is one of the utmost safety concerns at highway-rail grade crossings (HRGCs) but many trespassing incidents have not been recorded and deeply studied since no collision or otherwise injured or killed, which might possibly contribute to crashes if they repeatedly occur. Detection and prevention of such events are critical for railroad safety improvements, while this task is challenging due to the immense labor costs required for processing streamed video files. This study developed an advanced You Look Only Once (YOLO) deep learning architecture and the Deep Simple Online and Real-time Tracking (Deep SORT) algorithm with a low confidence tracking filter for real-time trespassing violation detection. Different types of trespassing violation were detected at a gated HRGC in Folkston, Georgia. 436 trespassing violations were identified in the selected 104-hour video data. The automated trespassing violation detection speed ranged from 43.2 to 654.5 frames per second (FPS), exceeding the field video data recording rate at 30 FPS. The developed methodology resulted in 32 false negatives and 20 false positive detections, with the precision, recall, and F1 values scoring above 92.0%. This work could assist railroad agencies in reducing trespassing violations based on real-time detection and tracking.