Neighbor-Guided Pseudo-Label Generation and Refinement for Single-Frame Supervised Temporal Action Localization.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Single-Frame Temporal Action Localization (SF-TAL) methods rely on threshold-based pseudo-labeling.
• Existing SF-TAL approaches exhibit inefficient data utilization and suffer from pseudo-label noise due to annotation variability and unreliable predictions.

Purpose of the Study:

• To enhance Single-Frame Temporal Action Localization (SF-TAL) by addressing inefficient data utilization and pseudo-label noise.
• To introduce novel strategies for generating and refining pseudo-labels in SF-TAL.

Main Methods:

• Proposed temporal neighbor-guided soft pseudo-label generation (TNPG) using a transformer encoder with local-global self-attention.
• Developed semantic neighbor-guided pseudo-label refinement (SNPR) by leveraging cosine similarity in feature space to identify and utilize semantic nearest neighbors.
• Integrated TNPG and SNPR to generate refined soft pseudo-labels for improved model training.

Main Results:

• Achieved state-of-the-art performance on THUMOS14, ActivityNet1.2, and ActivityNet1.3 datasets.
• Demonstrated significant performance improvements through comprehensive experimental validation.
• The proposed methods effectively utilize unlabeled frames and mitigate pseudo-label noise.

Conclusions:

• The proposed TNPG and SNPR strategies significantly improve SF-TAL performance.
• Leveraging temporal and semantic neighbor relationships enhances pseudo-label quality and model training.
• The approach offers a robust solution for temporal action localization challenges.