Real-time visual inspection system for grading fruits using computer vision and deep learning techniques

Abstract

Traditional manual visual grading of fruits has been one of the important challenges faced by the agricultural industry due to its laborious nature as well as inconsistency in inspection and classification process. Automated defects detection using computer vision and machine learning has become a promising area of research with a high and direct impact on the domain of visual inspection. In this study, we propose an efficient and effective machine vision system based on the state-of-the-art deep learning techniques and stacking ensemble methods to offer a non-destructive and cost-effective solution for automating the visual inspection of fruits’ freshness and appearance. We trained, tested and compared the performance of various deep learning models including ResNet, DenseNet, MobileNetV2, NASNet and EfficientNet to find the best model for the grading of fruits. The proposed system also provides a real time visual inspection using a low cost Raspberry Pi module with a camera and a touchscreen display for user interaction. The real time system efficiently segments multiple instances of the fruits from an image and then grades the individual objects (fruits) accurately. The system was trained and tested on two data sets (apples and bananas) and the average accuracy was found to be 99.2% and 98.6% using EfficientNet model for apples and bananas test sets, respectively. Additionally, a slight improvement in the recognition rate (0.03% for apples and 0.06% for bananas) was noted while applying the stacking ensemble deep learning methods. The performance of the developed system has been found higher than the existing methods applied to the same data sets previously. Further, during real-time testing on actual samples, the accuracy was found to be 96.7% for apples and 93.8% for bananas which indicates the efficacy of the developed system.