Pothole detection in adverse weather: leveraging synthetic images and attention-based object detection methods

Abstract

Abstract Potholes are a pervasive road hazard with the potential to cause accidents and vehicle damage. Detecting potholes accurately is essential for timely repairs and ensuring road safety. However, existing detection methods often struggle to perform in adverse weather conditions, including rain, snow, and low visibility. This work aims to improve pothole detection across diverse weather and lighting scenarios, employing a two-phase strategy that integrates data augmentation with images generated by Generative Adversarial Networks (GANs) and the deployment of visual attention techniques. For this purpose, advanced models such as YOLOv8, RT-DETR, and our modified version of YOLOv8 were employed. In the first phase, multiple image-to-image translation models were trained and applied to a real-world dataset to generate synthetic images of potholes under different weather conditions, including rain, fog, overcast, dawn, and night. The detection accuracy results show improvements in all monitored metrics across most tested conditions following the incorporation of augmentation. The most significant improvement resulting from augmentation was observed in low-visibility conditions, captured during evening and night, with an increase of up to 11% and 19% in mean Average Precision (mAP@.5) across all models. The second phase employed different modifications of YOLOv8 with modules such as Attention-Based Dense Atrous Spatial Pyramid Pooling, Vision Transformer and Global Attention Mechanism to enhance the detection of potholes in challenging visual conditions. The compensation for increased model complexity, such as the utilization of depthwise convolutions, was also employed. To evaluate the effectiveness of this approach, a publicly available pothole dataset with images captured in diverse weather conditions is used. The results indicate that the proposed method achieved an 8.4% improvement pre-augmentation and a 5.3% improvement post-augmentation compared to the original YOLOv8, surpassing existing approaches in terms of accuracy and enhancing pothole detection in adverse weather conditions.