Synthesizing Knowledge-Enhanced Features for Real-World Zero-Shot Food Detection

Area of Science:

• Computer Vision
• Artificial Intelligence
• Food Science

Background:

• Food computing utilizes computer vision for food analysis, crucial for nutrition and health applications.
• Zero-Shot Detection (ZSD) is essential for identifying novel food items in real-world scenarios like smart kitchens.
• Existing ZSD methods struggle with fine-grained food detection due to inter-class similarity and complex semantic attributes.

Purpose of the Study:

• To benchmark Zero-Shot Food Detection (ZSFD) using the new FOWA dataset with attribute annotations.
• To propose a novel framework, ZSFDet, addressing fine-grained challenges in ZSFD.
• To enhance the distinction of various food categories by exploiting attribute interactions.

Main Methods:

• Introduced the FOWA dataset for Zero-Shot Food Detection (ZSFD) research.
• Proposed ZSFDet framework utilizing multi-source graphs to model food category-attribute correlations.
• Developed Knowledge-Enhanced Feature Synthesizer (KEFS) with a region feature diffusion model for fine-grained feature generation.

Main Results:

• ZSFDet achieved superior performance on the FOWA and UECFOOD-256 datasets.
• Demonstrated significant improvements in Zero-Shot Detection mean Average Precision (mAP) by 1.8% and 3.7% over the RRFS baseline.
• Showcased enhanced performance on general ZSD tasks using PASCAL VOC and MS COCO datasets.

Conclusions:

• ZSFDet effectively tackles fine-grained problems in ZSFD by exploiting attribute interactions.
• The integration of multi-source knowledge enhances feature representation for improved food detection.
• The proposed approach offers a promising direction for advancing intelligent food analysis systems.