Weakly Supervised Temporal Action Localization Through Contrast Based Evaluation Networks.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Weakly-supervised temporal action localization (WS-TAL) uses video-level labels for action detection.
• WS-TAL is cost-effective but struggles with ambiguous localization in untrimmed videos.
• Existing WS-TAL methods often treat action localization as a post-processing step.

Purpose of the Study:

• To develop a novel WS-TAL method that overcomes localization ambiguities.
• To introduce a new approach for generating fine-grained pseudo supervision using temporal contrasts.
• To enable end-to-end training for WS-TAL by integrating localization into the network.

Main Methods:

• Proposed Contrast-based Localization EvaluAtioN Network (CleanNet).
• Introduced a temporal action proposal evaluator leveraging snippet-level prediction contrasts.
• Developed an integral action localization module for end-to-end training.
• Explored temporal contrast for temporal action proposal generation under weak supervision.

Main Results:

• CleanNet effectively resolves localization uncertainty by evaluating temporal contrast scores.
• The integrated localization module enables end-to-end training, outperforming post-processing methods.
• Achieved state-of-the-art performance on THUMOS14, ActivityNet v1.2, and v1.3 datasets.
• Demonstrated the first successful application of temporal contrast for TAP generation in WS-TAL.

Conclusions:

• Temporal contrasts are indicative of action boundaries, providing valuable supervisory signals.
• CleanNet offers a robust and efficient solution for WS-TAL.
• The proposed method advances the field of weakly-supervised video understanding and action localization.