Uncertainty Guided Collaborative Training for Weakly Supervised and Unsupervised Temporal Action Localization.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Weakly supervised action localization (WSAL) and unsupervised temporal action localization (UAL) aim to identify actions and their temporal boundaries using limited training data.
• Attention-based methods excel by focusing on relevant action segments, but their performance is limited by the lack of segment-level supervision for attention weights.

Purpose of the Study:

• To propose a novel Uncertainty Guided Collaborative Training (UGCT) strategy to enhance attention-based methods for WSAL and UAL.
• To address the challenge of noisy attention weights in existing methods by introducing a robust training framework.

Main Methods:

• Developed an online pseudo-label generation module where RGB and FLOW streams collaborate.
• Integrated an uncertainty-aware learning module to mitigate noise in pseudo-labels.
• Employed a collaborative training strategy exchanging information between RGB and FLOW streams.

Main Results:

• Demonstrated significant performance improvements on benchmark datasets (THUMOS14, etc.).
• Achieved over 7.0% performance gain in mean Average Precision (mAP) at IoU=0.5 on the THUMOS14 dataset.
• Validated the effectiveness across three different attention-based methods.

Conclusions:

• The proposed UGCT strategy effectively enhances WSAL and UAL performance by improving attention quality.
• Collaborative learning between RGB and FLOW streams, combined with uncertainty mitigation, leads to more accurate action localization.
• UGCT offers an efficient and effective approach to overcome limitations in current attention-based temporal action localization models.