Synthetic Data-Driven Real-Time Detection Transformer Object Detection in Raining Weather Conditions

Abstract

Images captured in rainy weather conditions often suffer from contamination, resulting in blurred or obscured objects, which can significantly impact detection performance due to the loss of identifiable texture and color information. Moreover, the quality of the detection model plays a pivotal role in determining detection outcomes. This study adopts a dual perspective, considering both pre-trained models and training data. It employs 15 image augmentation techniques, combined with neural style transfer (NST), CycleGAN, and an analytical method, to synthesize images under rainy conditions. The Real-Time Detection Transformer (RTDETR) and YOLOv8 pre-trained models are utilized to establish object detection frameworks tailored for rainy weather conditions. Testing is carried out using the DAWN (Detection in Adverse Weather Nature) dataset. The findings suggest compatibility between the pre-trained detection models and various data synthesis methods. Notably, YOLOv8 exhibits better compatibility with CycleGAN data synthesis, while RTDETR demonstrates a stronger alignment with the NST and analytical approaches. Upon the integration of synthesized rainy images into model training, RTDETR demonstrates significantly enhanced detection accuracy compared to YOLOv8, indicating a more pronounced improvement in performance. The proposed approach of combining RTDETR with NST in this study shows a significant improvement in Recall (R) and mAP50-95 by 16.35% and 15.50%, respectively, demonstrating the robust rainy weather resilience of this method. Additionally, RTDETR outperforms YOLOv8 in terms of inference speed and hardware requirements, making it easier to use and deploy in real-time applications.