Video-based Intake Gesture Recognition Using Meal-length Context.

Area of Science:

• Biomedical Engineering
• Computer Vision
• Nutritional Science

Background:

• Eating behaviors significantly impact global health, contributing to major diseases.
• Automated video analysis offers potential for objective monitoring of dietary intake.
• Existing methods using short video clips miss meal-length patterns, limiting accuracy.

Purpose of the Study:

• To develop a novel video analysis pipeline for monitoring eating behaviors using the entire meal context.
• To improve the accuracy of automated detection of food intake gestures (bite and drink).
• To address limitations of short-clip analysis in deep learning models for eating behavior monitoring.

Main Methods:

• Proposed a novel pipeline analyzing the entire meal context (5-40 minutes) for eating behavior monitoring.
• Developed a global detector framework to learn meal-length patterns with efficient computation.
• Introduced a new augmentation technique to generate extensive meal-length feature samples for training.

Main Results:

• The pipeline significantly enhances state-of-the-art window-based networks in gesture detection.
• Demonstrated substantial reduction in false positives for bite and drink gesture detection.
• Achieved high F1 scores (0.93 for bite, 0.88 for drink) on the large Clemson Cafeteria dataset.

Conclusions:

• Analyzing entire meal context improves automated eating behavior monitoring accuracy.
• The novel pipeline and augmentation technique offer a more effective approach to gesture detection.
• This method holds promise for objective dietary assessment in health research.