Motion-Driven Visual Tempo Learning for Video-Based Action Recognition.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Action visual tempo is crucial for distinguishing similar human actions based on dynamics and temporal scale.
• Existing methods for capturing visual tempo are computationally expensive or miss fine-grained temporal details.

Purpose of the Study:

• To propose a novel Temporal Correlation Module (TCM) for efficient and effective extraction of action visual tempo.
• To integrate TCM into existing action recognition backbones as a plug-in module.

Main Methods:

• The proposed Temporal Correlation Module (TCM) comprises a Multi-scale Temporal Dynamics Module (MTDM) and a Temporal Attention Module (TAM).
• MTDM uses correlation operations for pixel-wise temporal dynamics across fast and slow tempos.
• TAM adaptively weights features by analyzing global information across different tempos.

Main Results:

• The TCM effectively extracts action visual tempo from low-level backbone features at a single layer.
• Experiments on benchmarks like Something-Something V1&V2, Kinetics-400, UCF-101, and HMDB-51 show significant performance improvements.
• The TCM demonstrably enhances existing video-based action recognition models.

Conclusions:

• The Temporal Correlation Module (TCM) offers a computationally efficient and effective approach to action visual tempo analysis.
• TCM significantly boosts the performance of current action recognition systems.
• The proposed method provides a valuable tool for advancing human action recognition research.