FruVeg_MultiNet: A hybrid deep learning-enabled IoT system for fresh fruit and vegetable identification with web interface and customized blind glasses for visually impaired individuals

Abstract

The automatic identification of fresh vegetables and fruits is imperative to streamline agricultural processes, ensuring rapid and accurate assessment of produce quality, reducing economic pressure, and addressing societal needs, particularly for visually impaired individuals. This research presents a pioneering approach for fresh fruit and vegetable identification through IoT and a hybrid deep learning model, combining EfficientNetB7 and ResNet50 architectures. The proposed hybrid model demonstrates remarkable accuracy, achieving 99.92% and 95.93% precision on dataset1 and dataset2, respectively. The study encompasses a comprehensive evaluation of four initial models: EfficientNetB7, VGG16, ResNet50, and VGG19. The hybrid model, which combines the best of these, performed better than the others. In addition to its high accuracy, the system achieved an average response time of 1.201 s, highlighting its efficiency in processing and decision-making. Considering these challenges in the agricultural industry, the research extends to fruit and vegetable classification, offering applications in self-service fruit or vegetable purchasing, production lines, and smart agriculture. Additionally, the societal impact is considered, with the development of technology aiding the visually impaired in assessing produce freshness. Furthermore, we developed a useful web application that categorizes fruits and vegetables and links to a detailed database offering important information about the recognized produce. • Deep learning approaches are applied for feature extraction and fine-tuning. • EfficientNetB7, VGG16, ResNet50, and VGG19 are utilized to find the best tuning model. • Combining EfficientNetB7 and ResNet50 produces 99.92% accuracy for the first dataset. • Produces an IoT system for blind and visually impaired (BVI) individuals. • Developed a web interface for fruit and vegetable classification in detail.