Learnable Feature Augmentation Framework for Temporal Action Localization.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Temporal action localization (TAL) performance is limited by the scarcity of annotated untrimmed video data.
• Existing methods struggle to effectively utilize available video data for accurate action recognition and localization.

Purpose of the Study:

• To enhance the utilization of existing video data for temporal action localization through feature augmentation.
• To develop a novel framework that improves the robustness and generalization of action detection models.

Main Methods:

• Feature augmentation using a Mask-based Feature Augmentation Module (MFAM) to generate diverse video feature views.
• MFAM captures temporal-semantic relationships, preserves essential action information, and ensures diversity through information recycling.
• Jointly training action detectors with original and augmented features for classification and localization.

Main Results:

• The proposed framework significantly improves the performance of various temporal action localization models.
• State-of-the-art results were achieved on four benchmark datasets for temporal action localization.
• The MFAM module can be removed during testing, incurring no additional computational overhead.

Conclusions:

• The Mask-based Feature Augmentation Module (MFAM) effectively addresses data limitations in temporal action localization.
• The framework enhances model robustness and generalization by learning richer video representations.
• This approach offers a computationally efficient method for advancing state-of-the-art performance in temporal action localization.