Semantic and Temporal Contextual Correlation Learning for Weakly-Supervised Temporal Action Localization.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Weakly-supervised temporal action localization (WSTAL) identifies and localizes actions in videos using only video-level labels.
• Key challenges include accurately discovering action categories and precisely localizing action instances within untrimmed videos.

Purpose of the Study:

• To address the limitations of existing WSTAL methods by proposing a novel network that jointly models semantic and temporal contextual correlations.
• To achieve both accurate action discovery and complete action localization through a unified dynamic correlation-embedding paradigm.

Main Methods:

• Introduced the Semantic and Temporal Contextual Correlation Learning Network (STCL-Net) comprising Semantic Correlation Learning (SCL) and Temporal Contextual Correlation Learning (TCL) modules.
• These modules model inter- and intra-video correlations for each snippet, enabling joint semantic and temporal understanding.

Main Results:

• STCL-Net demonstrated superior or comparable performance against state-of-the-art models across multiple benchmarks.
• Achieved significant improvements, including a 7.2% increase in average mean Average Precision (mAP) on the THUMOS-14 dataset.

Conclusions:

• The proposed STCL-Net effectively addresses the challenges in WSTAL by jointly modeling semantic and temporal contextual correlations.
• Ablation studies confirmed the effectiveness and robustness of individual components within the STCL-Net architecture.