Breaking Winner-Takes-All: Iterative-Winners-Out Networks for Weakly Supervised Temporal Action Localization

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Weakly supervised temporal action localization (WS-TAL) is challenging due to limited annotations.
• Unconstrained web videos often lack precise frame-level action labels.
• Existing methods may suffer from the 'winner-takes-all' phenomenon, neglecting less discriminative action instances.

Purpose of the Study:

• To develop a novel network for WS-TAL using only video-level action labels.
• To address the 'winner-takes-all' issue in existing WS-TAL approaches.
• To improve the localization of both discriminative and less discriminative action instances.

Main Methods:

• Propose an iterative 'winners-out' training strategy to select and remove discriminative instances progressively.
• Devise a class-discriminative localization technique using attention mechanisms to identify target action instances.
• Integrate these components into an end-to-end network for WS-TAL without frame-level annotations.

Main Results:

• The proposed method outperforms state-of-the-art weakly supervised approaches on ActivityNet1.3.
• Achieved an mAP improvement from 16.9% to 20.5% on THUMOS14.
• Demonstrated comparable accuracy to methods using frame-level supervision, despite using only weak video-level labels.

Conclusions:

• The iterative 'winners-out' strategy effectively alleviates the 'winner-takes-all' problem in WS-TAL.
• The class-discriminative localization technique enhances the selection of action instances.
• The proposed end-to-end network offers a powerful solution for WS-TAL with weak supervision.