Beyond the Naked Eye: Computer Vision for Detecting Brown Marmorated Stink Bug and Its Punctures

Lennart Almstedt, Francesco Betti Sorbelli, Bas Boom, Rosalba Calvini, Elena Costi, Alexandru Dinca, Veronica Ferrari, Daniele Giannetti, Loretta Ichim, Amin Kargar,

IEEE Transactions on AgriFood Electronics · 2024 · 12 citations · 34 references

DOIFull text

Open access

Concepts

Abstract

In this article, we introduce machine learning (ML) techniques developed for the monitoring of the brown marmorated stink bug (BMSB), a significant agricultural pest responsible for considerable crop damage worldwide. The <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">Haly.ID</small> project, initiated in early 2021, aims to enhance BMSB monitoring through the utilization of information and communication technology methods. We employ computer vision techniques on RGB images captured by drones and investigate the performance of deep neural networks to evaluate the impact of this invasive species on crop yields in orchards around Europe. Specifically, we evaluate the single shot multibox detector, detection transformer, <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">YOLOv5</small> , <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">YOLOv9</small> , and <sc xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">YOLOv10</small> architectures for full-level and patch-level image analysis, respectively. To improve detection accuracy, we experiment with shortwave infrared hyperspectral imaging (SWIR-HSI) in laboratory settings. Given that pheromone baited traps are the most accepted tools for pest detection by field operators, we also propose an Internet of Things sticky trap with an integrated camera equipped with lightweight convolutional neural networks models operating “on the edge” in this resource constrained system. In addition, we develop a client–server application for real-time bug detection, integrating the ML models to provide accessible results to farmers. Lastly, we explore effective postharvesting strategies using SWIR-HSI images to detect insect punctures invisible to the naked eye, thereby enhancing the quality of marketable fruit.

References

34