Multi-Hierarchical Category Supervision for Weakly-Supervised Temporal Action Localization.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Weakly Supervised Temporal Action Localization (WTAL) uses video-level labels for action localization in untrimmed videos.
• Current WTAL models often miss common sub-actions, leading to incomplete action segment localization.
• Existing methods focus on complex architectures to capture more complete actions.

Purpose of the Study:

• To introduce a novel supervision method, multi-hierarchical category supervision (MHCS), to identify more sub-actions, including common ones.
• To address the issue of incomplete action localization caused by ignoring shared sub-actions in WTAL.
• To propose a model-agnostic and non-intrusive approach applicable to existing WTAL methods.

Main Methods:

• Developed a multi-hierarchical category supervision (MHCS) method.
• Utilized hierarchical clustering to group action categories with similar sub-actions into super-classes.
• Trained models using these super-classes to encourage attention to common sub-actions.

Main Results:

• The MHCS method successfully encouraged models to focus on common sub-actions.
• MHCS improved the performance of four state-of-the-art WTAL methods.
• Significant performance gains were observed on THUMOS14, ActivityNet1.2, and ActivityNet1.3 datasets.

Conclusions:

• MHCS is an effective supervision method for improving WTAL by capturing more complete action segments.
• The proposed approach enhances action localization accuracy without altering existing model architectures.
• MHCS offers a valuable contribution to the field of weakly supervised learning for video analysis.