A Temporal-Aware Relation and Attention Network for Temporal Action Localization.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Temporal action localization is crucial for smart surveillance but computationally intensive.
• Existing methods struggle with classifying actions and precisely locating their start and end times in untrimmed videos.
• High computational costs hinder the training and inference of current temporal action localization models.

Purpose of the Study:

• To propose a novel Temporal-Aware Relation and Attention Network (TRA) for efficient and accurate temporal action localization.
• To develop an anchor-free, end-to-end architecture that fully leverages temporal-aware information.
• To reduce the computational resources required for training and inference in temporal action localization.

Main Methods:

• A temporal self-attention module was designed to weigh features within actions by understanding temporal relationships.
• A multiple temporal aggregation module was constructed to consolidate temporal domain information.
• A graph relation module refined action boundaries and classifications using aggregated graph features, all within a unified framework.

Main Results:

• The proposed TRA method achieved state-of-the-art performance on the THUMOS14 dataset with an average mAP of 67.6%.
• Comparable results were obtained on the ActivityNet1.3 dataset, with an average mAP of 34.4%.
• TRA demonstrated significant improvements over existing methods like A2Net, PCG-TAL, and AFSD on the THUMOS14 dataset.

Conclusions:

• The TRA network offers a computationally efficient and highly accurate solution for temporal action localization.
• The unified framework effectively utilizes temporal awareness for improved action recognition and localization.
• The proposed method sets a new benchmark for performance in temporal action localization tasks.