Improved Detection of Multi-Class Bad Traffic Signs Using Ensemble and Test Time Augmentation Based on Yolov5 Models

Abstract

Various factors such as natural disasters, vandalism, weather, and environmental conditions can affect the physical state of traffic signs. The proposed model aims to improve detection of traffic signs affected by partial occlusion as a result of overgrown vegetation, displaced signs (those knocked down, bent), perforated signs (those damaged with holes), faded signs (color degradation), rusted signs (corroded surface), and de-faced signs (placing graffiti, etc., by vandals). This research aims to improve the detection of bad traffic signs using three approaches. In the first approach, Spiral Pooling Pyramid-Fast (SPPF) and C3TR modules are introduced to the architecture of Yolov5 models. SPPF helps provide a multi-scale representation of the input feature map by pooling at different scales, which is useful in improving the quality of feature maps and detecting bad traffic signs of various sizes and perspectives. The C3TR module uses convolutional layers to enhance local feature extraction and transformers to boost understanding of the global context. Secondly, we use predictions of Yolov5 as base models to implement a mean ensemble to improve performance. Thirdly, test time augmentation (TTA) is applied at test time by using scaling and flipping to improve accuracy. Some signs are generated using stable diffusion techniques to augment certain classes. We test the proposed models on the CCTSDB2021, TT100K, GTSDB, and GTSRD datasets to ensure generalization and use k-fold cross-validation to further evaluate the performance of the models. The proposed models outperform other state-of-the-art models in comparison.