Watermelon Detection and Localization Technology Based on GTR-Net and Binocular Vision

Abstract

Fruit and melon picking, a labor-intensive task facing rising costs, prompts the need for technological solutions. While picking robots aim to address this, the complex growth environment poses challenges in identifying and locating fruits due to factors like light and leaf occlusion. This study focuses on designing a recognition and localization method tailored to the natural growth conditions of melons and fruits, aiming to provide precise positional information for effective harvesting. Leveraging GTR-Net and binocular stereo vision, the proposed technology integrates a lightweight backbone network with Ghost bottleneck and TCSPG modules. The inclusion of TCSPRep and RepBlock modules enhances feature fusion, adapting to varying lighting conditions. To tackle occlusion challenges, the study introduces the RIoU loss function. Experimental validation using watermelons demonstrates GTR-Net’s adaptability, achieving a remarkable mean Average Precision (mAP) of 91.7%. The model, with a compact 10.3MB size, attains a high detection speed of 105.96 frames per second (FPS), meeting real-time detection requirements. Our research enhances robot adaptability in complex environments, offering valuable insights for watermelon identification by mobile harvesting robots in challenging conditions.