A Two-Level Food Classification System For People With Diabetes Mellitus Using Convolutional Neural Networks

Accurate estimation of food's macronutrient content for people with Diabetes Mellitus (DM) is of great importance, as it determines postprandial insulin dosage. This paper introduces a classification system for food images that is adjusted to the nutritional needs of people with DM. A two-level image classification scheme, exploiting Convolutional Neural Networks (CNNs), is proposed, in order to classify an image in one of eight broad food categories with similar macronutrient content and then assign it to a specific food within that category. To this end, a visual dataset, namely NTUA-Food 2017, has been designed, consisting of 3248 images organized in eight broad food categories of totally 82 different foods. Moreover, a novel evaluation metric is proposed, which penalizes classification errors proportionally to the discrepancy in postprandial blood sugar levels between the actual and predicted class. The proposed system achieves 84.18% and 85.94% classification accuracy at the first and second level of classification, respectively, on the NTUA-Food 2017 dataset. The algorithm developed for the first level of classification on the NTUA-Food 2017 dataset improves classification accuracy on the benchmark Food Image Dataset (FID) to 97.08% outperforming previous approaches. The algorithm's mean error in terms of carbohydrate content estimation on the NTUA-Food 2017 dataset is less than 2 g per food serving.