Temporal Hierarchical Dictionary Guided Decoding for Online Gesture Segmentation and Recognition.

Area of Science:

• Computer Vision
• Human-Computer Interaction
• Machine Learning

Background:

• Online gesture recognition is difficult due to incomplete movement data and early ambiguity.
• Existing methods struggle with local optima in recognizing partially performed gestures.

Purpose of the Study:

• To develop a robust online gesture recognition system that overcomes the challenges of incomplete and ambiguous movements.
• To improve the accuracy and efficiency of Hidden Markov Model (HMM) decoding for real-time gesture recognition.

Main Methods:

• A temporal hierarchical dictionary guides the Hidden Markov Model (HMM) decoding process.
• A novel 'relative entropy map' (REM) quantifies temporal uncertainty to aid HMM decoding.
• A progressive learning strategy enables iterative learning of robust HMM states using neural networks.

Main Results:

• The proposed method achieves state-of-the-art performance on three challenging gesture recognition databases.
• The framework effectively extracts discriminative gesture features while reducing redundancy in HMM transitions.
• Demonstrated successful online recognition of continuous gesture streams, even when gestures are only partially performed.

Conclusions:

• The developed framework significantly enhances online skeleton-based gesture recognition accuracy and robustness.
• Relative entropy maps provide a valuable mechanism for guiding HMM decoding in ambiguous temporal contexts.
• The progressive learning strategy contributes to more stable and accurate HMM state recognition.