Fine-grained food image classification and recipe extraction using a customized deep neural network and NLP

Area of Science:

• Computer Science
• Artificial Intelligence
• Computational Linguistics

Background:

• Increasing global health issues linked to eating habits are driving interest in mindful eating.
• The application of deep learning to food-related data is gaining traction for analysis and understanding.
• Challenges in food image recognition, such as intra-class variability and inter-class similarity, require advanced solutions.

Purpose of the Study:

• To develop an integrated framework for classifying food images and automating recipe extraction.
• To address the limitations of existing models in handling complex food image datasets.
• To improve the accuracy and efficiency of food data analysis for dietary insights.

Main Methods:

• A customized lightweight deep convolutional neural network, MResNet-50, was developed for food image classification.
• Natural language processing algorithms, Word2Vec and Transformers, were employed for automated ingredient and recipe extraction.
• A semi-structured domain ontology was constructed to map relationships between cuisine, food items, and ingredients.

Main Results:

• The proposed framework achieved improved accuracy on the Food-101 and UECFOOD256 datasets, with increases of 2.4% and 7.5%, respectively.
• The model demonstrated superior performance compared to existing methods like DeepFood, CNN-Food, and Wiser.
• Successful automated extraction of ingredients and recipes was achieved, demonstrating the framework's practical utility.

Conclusions:

• The novel framework effectively integrates deep learning and NLP for enhanced food image classification and recipe extraction.
• The approach offers a significant advancement in analyzing food-related data, addressing key challenges in the field.
• This work provides a foundation for developing more sophisticated AI-driven tools for dietary analysis and mindful eating initiatives.