Related Experiment Video
Updated: Oct 17, 2026
Deep Neural Networks for Image-Based Dietary Assessment
Published on: March 13, 2021
Developing a Machine Learning-Based System for Nutrient Profiling of Pakistani Cuisine
Alishba Umer Farooq1, Fiza Khan1, Huda Ijaz1
1Faculty of Computing and Information Technology, The University of Punjab, Lahore P.O. Box 54590, Pakistan.
Background:
Chronic diseases such as diabetes and heart disease are rising in South Asia, with poor diet a major contributor. Most dietary apps are designed for Western foods and struggle with culturally diverse dishes such as Pakistani cuisine.
Objective:
This study aimed to build a machine learning-based system that can recognise Pakistani dishes, estimate portion sizes, and calculate their nutritional value.
Methods:
We developed a multi-stage automated dietary assessment pipeline for Pakistani cuisine using a multi-source dataset covering 83 dish classes across 10 categories. Following data cleaning, class balancing, and targeted augmentation, deep learning architectures were evaluated for food classification and segmentation. ResNet-50, MobileNetV2, and YOLOv8 were used for food recognition, while YOLOv8-Seg and Mask R-CNN enabled pixel-level segmentation for portion and nutrient estimation. All models were optimized using transfer learning and evaluated using standard classification and mask-quality metrics, followed by mobile application usability testing.
Results:
A multi-source dataset of ten categories and 83 Pakistani dishes was used to train and compare deep learning models for food classification and segmentation. Portion size was estimated using a segmentation- and depth-based approach without a reference object, while nutritional values were derived from standardised recipes and established food databases. The models were integrated into a mobile application for usability testing. YOLO-based models achieved the best balance of accuracy and speed for real-time classification and segmentation. Portion estimates were reasonably close to measured values, and usability testing showed the app was easy to use, with minor improvements suggested.
Conclusions:
Overall, this work demonstrates the feasibility of an AI-based dietary assessment system adapted to Pakistani cuisine, with future work focused on reducing processing demands and improving depth-based portion estimation.
