Adaptive YOLOv6 with spatial Transformer Networks for accurate object detection and Multi-Angle classification in remote sensing images

Abstract

In the field of contemporary remote sensing image analysis, deep learning techniques have emerged as a recognized and revolutionary standard. However, these images often feature complex backgrounds and objects positioned at unusual angles. In this study, we address the challenges posed by significant background distortion and unique angles by proposing a window function-driven dynamic matching and smooth label approach. We integrate You Only Look Once (Yolo) version 6 model with Spatial Transformer Networks (STNet) to form Yolo-STNet. This combination leverages Yolov6′s strengths in precise bounding box prediction and multi-orientation object detection. An additional module, STNet, is employed to handle multi-angle classification, adaptively enhancing the classifier’s performance in response to identified angles. Even in complex and crowded environments, Yolo-STNet improves the model’s ability to accurately detect, classify, and locate objects at different orientations. We evaluated the proposed Yolo-STNet’s performance using the Object Detection in Aerial Images (DOTA) dataset, employing three metrics: angle classification accuracy , F1-score, and Average Precision at IoU with threshold of 0.50 (AP50). The results demonstrated that Yolo-STNet outperformed other approaches and showed significant improvements. For instance, Yolo-STNet significantly outperformed Faster Region-Based Convolutional Neural Network and Oriented Region-Based Convolutional Neural Network in terms of AP50 on the DOTA dataset, with gains of 5.59% and 4.28%, respectively. This highlights the effectiveness of the proposed Yolo-STNet as a tool for remote sensing applications .