Non-Local Temporal Difference Network for Temporal Action Detection.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Temporal action detection (TAD) is crucial for video understanding, aiming to identify action instances within untrimmed videos.
• Existing methods often struggle with redundant frame information and weakened distant features due to stacked convolutions.
• Varying action durations pose challenges for single-scale modeling in complex video structures.

Purpose of the Study:

• To develop an advanced temporal action detection method that overcomes limitations of existing approaches.
• To effectively model long-range temporal dependencies and handle diverse action instance durations.
• To enhance motion information and boundary feature extraction for improved accuracy.

Main Methods:

• Proposed a non-local temporal difference network (NTD) comprising three key modules: Chunk Convolution (CC), Multiple Temporal Coordination (MTC), and Temporal Difference (TD).
• The TD module utilizes temporal attention to enhance motion and boundary features.
• The CC module extracts features from distant frames by processing chunks independently, while the MTC module aggregates multi-scale temporal features efficiently.

Main Results:

• Achieved state-of-the-art performance on large-scale datasets.
• Demonstrated superior accuracy with 36.2% mAP@avg on ActivityNet-v1.3 and 71.6% mAP@0.5 on THUMOS-14.
• The NTD effectively captures long-range dependencies and handles variable action durations.

Conclusions:

• The proposed NTD significantly advances temporal action detection capabilities.
• The novel modules (CC, MTC, TD) offer an effective solution for modeling complex temporal dynamics in videos.
• This work provides a robust framework for more accurate and efficient video understanding.