Single-Stage Intake Gesture Detection Using CTC Loss and Extended Prefix Beam Search

Area of Science:

• Biomedical Engineering
• Computer Science
• Human-Computer Interaction

Background:

• Accurate detection of individual intake gestures is crucial for automatic dietary monitoring.
• Current advanced methods utilize a two-stage approach combining deep neural networks with frame-level probability analysis.
• Both inertial sensor data and video data have been explored for capturing upper body movements during eating and drinking.

Purpose of the Study:

• To propose and evaluate a novel single-stage approach for direct decoding of intake gesture probabilities from sensor data.
• To enhance the accuracy and efficiency of automatic dietary monitoring systems.
• To simplify the training requirements for intake gesture detection models.

Main Methods:

• Developed a single-stage deep learning model for end-to-end intake gesture detection.
• Employed Connectionist Temporal Classification (CTC) loss for weakly supervised training.
• Utilized a novel extended prefix beam search decoding algorithm for sparse event detection.

Main Results:

• Achieved relative F1 score improvements of 1.9% to 6.2% over the two-stage approach across two datasets.
• Demonstrated improved performance for both intake detection and eating/drinking detection tasks.
• Validated the approach using both video and inertial sensor data.

Conclusions:

• The proposed single-stage approach offers improved detection performance for dietary intake gestures.
• This method simplifies training data requirements and enhances the overall efficiency of automatic dietary monitoring.
• The novel CTC loss and decoding algorithm contribute to more accurate and robust intake gesture recognition.