Weed database development: An updated survey of public weed datasets and cross-season weed detection adaptation

Abstract

Weeds are a major threat to crop production. Automated innovations for reducing herbicides and labor needed for weeding have become a high priority for sustainable weed management. The current state-of-the-art weeding systems still cannot reliably recognize weeds in changing field conditions for precision weed control. Enhancing weed recognition accentuates the critical need to develop dedicated, labeled weed databases and whereby train advanced AI (artificial intelligence) models while ensuring the robustness of models across diverse field conditions. This study presents an up-to-date survey on publicly available image datasets for weed recognition. Among 36 datasets identified, limitations exist in terms of data variations and distribution shifts, and few of the datasets are suitable for examining the robustness of weed recognition models. A new two-season weed dataset is described in this study, with two sub-datasets for the images collected in the seasons of 2021 and 2022, respectively. Three state-of-the-art deep learning object detectors, i.e., YOLOX, YOLOv8, and DINO, were benchmarked and evaluated for their in-season and cross-season weed detection performance on the dataset. All three models attained in-season detection accuracies of 92% and higher in terms of [email protected]. However substantial accuracy drops of up to 14.5% were observed between in-season and cross-season testing, especially for YOLOX and YOLOv8. Unsupervised domain adaptation based on an implicit instance-invariant network (I3Net) was investigated for improved generalization of the YOLO models. The I3Net-based models resulted in accuracy improvements of 1.4% and 3.3% for YOLOX and YOLOv8, respectively, compared to modeling without domain adaptation, in the cross-season testing. Both the two-season detection and software programs for weed detection modeling in this study will be made publicly available.