Adaptive Prototype Learning for Weakly-Supervised Temporal Action Localization.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Weakly-supervised Temporal Action Localization (WTAL) trains models using only video-level labels.
• Existing WTAL methods struggle with localization incompleteness and background interference.
• Current attention mechanisms have limitations in adapting to video diversity and separating foreground/background effectively.

Purpose of the Study:

• To address the limitations of existing WTAL methods.
• To propose a novel Adaptive Prototype Learning (APL) method.
• To improve the accuracy and robustness of temporal action localization.

Main Methods:

• Developed an Adaptive Transformer Network (ATN) to model background and learn video-adaptive prototypes.
• Introduced an OT-based Collaborative (OTC) training strategy using Optimal Transport (OT) for RGB and FLOW streams.
• Guided prototype learning and resolved foreground-background ambiguity.

Main Results:

• The proposed APL method effectively learns video-adaptive prototypes.
• APL successfully addresses localization incompleteness and background interference.
• Experiments on THUMOS14 and ActivityNet benchmarks show APL outperforms state-of-the-art methods.

Conclusions:

• APL offers a robust solution for weakly-supervised temporal action localization.
• The combination of ATN and OTC training significantly enhances localization performance.
• The method demonstrates superior performance on standard benchmarks, advancing the field of WTAL.