Nutrients · 2017 · 263 citations · 23 references
ComputingComputer ScienceNutritionComputer VisionDietary AssessmentHuman NutritionDeep LearningFood GroupsFood IntakeImage Feature EngineeringObject RecognitionClassification AccuracyClassification Accuracy AnalysisClassification Accuracy AssessmentDeep Learning-based ClassificationFood Group Intake EstimationFood Item ClassificationImage-based Detection
Food‑image recognition is essential for dietary assessment but remains difficult, and traditional methods achieve low accuracy, prompting the development of automatic systems. This study introduces NutriNet, a novel deep‑convolutional neural network designed to detect and recognize food and drink items. NutriNet was trained on 225,953 512×512 images of 520 items, achieving 86.72 % classification and 94.47 % detection accuracy, and includes an online component for continual fine‑tuning. In real‑world tests on smartphone images from Parkinson’s patients, NutriNet reached 55 % top‑five accuracy, improved the UNIMIB2016 baseline, and is deployed in a mobile app for dietary assessment.
Automatic food image recognition systems are alleviating the process of food-intake estimation and dietary assessment. However, due to the nature of food images, their recognition is a particularly challenging task, which is why traditional approaches in the field have achieved a low classification accuracy. Deep neural networks have outperformed such solutions, and we present a novel approach to the problem of food and drink image detection and recognition that uses a newly-defined deep convolutional neural network architecture, called NutriNet. This architecture was tuned on a recognition dataset containing 225,953 512 × 512 pixel images of 520 different food and drink items from a broad spectrum of food groups, on which we achieved a classification accuracy of 86.72%, along with an accuracy of 94.47% on a detection dataset containing 130,517 images. We also performed a real-world test on a dataset of self-acquired images, combined with images from Parkinson’s disease patients, all taken using a smartphone camera, achieving a top-five accuracy of 55%, which is an encouraging result for real-world images. Additionally, we tested NutriNet on the University of Milano-Bicocca 2016 (UNIMIB2016) food image dataset, on which we improved upon the provided baseline recognition result. An online training component was implemented to continually fine-tune the food and drink recognition model on new images. The model is being used in practice as part of a mobile app for the dietary assessment of Parkinson’s disease patients.
23
Deep Residual Learning for Image Recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren et al. · 2016 · 222.1K citations · Full text
Going deeper with convolutions
Christian Szegedy, Wei Liu, Yangqing Jia et al. · 2015 · 46.8K citations
Fully convolutional networks for semantic segmentation
Jonathan Long, Evan Shelhamer, Trevor Darrell · 2015 · 36.9K citations
Dropout: a simple way to prevent neural networks from overfitting
Nitish Srivastava, Geoffrey E. Hinton, Alex Krizhevsky et al. · 2014 · 34.3K citations