Real-Time Barge Detection Using Traffic Cameras and Deep Learning on Inland Waterways

Abstract

Inland waterways are critical for freight movement, but limited means exist for monitoring their performance and usage by freight-carrying vessels (e.g., barges). Although methods to track vessels (e.g., tug and tow boats) are publicly available through Automatic Identification System (AIS), ways to track freight tonnages and commodity flows carried on barges along these critical marine highways are nonexistent, especially in real-time settings. This study developed a method to detect barge traffic on inland waterways using existing traffic cameras with opportune viewing angles. Deep learning models You Only Look Once (YOLO), Single Shot MultiBox Detector (SSD), and EfficientDet were employed to detect the presence of vessels/barges from video and classify them (no vessel or barge, vessel without barge, vessel with barge, barge). A dataset of 331 annotated images was collected from five existing traffic cameras along the Mississippi and Ohio Rivers for model development. YOLOv8 achieved an F1-score of 96%, outperforming YOLOv5, SSD, and EfficientDet at 86%, 79%, and 77%, respectively. Sensitivity analysis was carried out for weather conditions (rain, fog) and location (Mississippi and Ohio River). A background subtraction technique normalized the video images across the various locations for the location sensitivity analysis. This model could be used to detect the presence of barges along river segments, which could be used for anonymous bulk commodity tracking and monitoring. Such data are valuable for long-range transportation planning efforts carried out by public transportation agencies, and for operational and maintenance planning conducted by federal agencies such as the U.S. Army Corps of Engineers.