Food Recognition: A New Dataset, Experiments, and Results

TLDR

Canteen tray images depict real dishes arranged in varied ways, with each tray containing multiple food class instances. The authors propose a new dataset to evaluate food recognition algorithms for dietary monitoring applications. The dataset includes 1027 trays with 3616 food instances across 73 classes, manually segmented, and is benchmarked with an automatic tray analysis pipeline that applies three classification strategies using various visual descriptors. Using CNN-based features, the benchmark achieves about 79 % food and tray recognition accuracy, and the dataset and framework are publicly available.

Abstract

We propose a new dataset for the evaluation of food recognition algorithms that can be used in dietary monitoring applications. Each image depicts a real canteen tray with dishes and foods arranged in different ways. Each tray contains multiple instances of food classes. The dataset contains 1027 canteen trays for a total of 3616 food instances belonging to 73 food classes. The food on the tray images has been manually segmented using carefully drawn polygonal boundaries. We have benchmarked the dataset by designing an automatic tray analysis pipeline that takes a tray image as input, finds the regions of interest, and predicts for each region the corresponding food class. We have experimented with three different classification strategies using also several visual descriptors. We achieve about 79% of food and tray recognition accuracy using convolutional-neural-networks-based features. The dataset, as well as the benchmark framework, are available to the research community.