A Hybrid RNN-HMM Approach for Weakly Supervised Temporal Action Segmentation.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Action recognition is a rapidly advancing field, but large-scale data demands impractical manual frame-based annotation.
• Weakly supervised learning offers a solution by leveraging alternative data sources, such as the order of actions.

Purpose of the Study:

• To develop a hierarchical approach for weakly supervised human action recognition using ordered action labels.
• To infer action start and end frames and train action classifiers without manual frame boundary annotation.

Main Methods:

• A hierarchical, coarse-to-fine strategy combining a framewise Recurrent Neural Network (RNN) model with coarse probabilistic inference.
• Iterative training of the RNN and probabilistic inference for temporal alignment of long sequences.
• Performance enhancement through approximating subaction counts and incorporating a regularizing length prior.

Main Results:

• The proposed system achieves competitive performance on temporal action segmentation and action alignment tasks.
• Evaluation on the Breakfast and Hollywood Extended datasets demonstrates the effectiveness of the approach.
• The combination of RNN and probabilistic inference enables accurate temporal alignment and classification.

Conclusions:

• The hierarchical approach effectively addresses weakly supervised learning of human actions from ordered labels.
• The method significantly reduces the need for impractical frame-based human annotation in action recognition.
• Further improvements are achieved by optimizing subaction approximation and length priors, offering a robust solution for action recognition challenges.