Learning SpatioTemporal and Motion Features in a Unified 2D Network for Action Recognition

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Current action recognition methods often rely on computationally expensive 3D CNNs and optical flows.
• These methods present challenges in terms of time and space efficiency.

Purpose of the Study:

• To develop a more efficient action recognition model using a unified 2D CNN framework.
• To represent both spatiotemporal and motion features effectively within a single network architecture.

Main Methods:

• Designed channel-wise spatiotemporal and motion modules for efficient feature extraction.
• Integrated these modules into a ResNet architecture, forming the STM network.
• Introduced a novel Twins Training framework with correlation loss and a siamese structure.

Main Results:

• The STM network effectively captures spatiotemporal and motion features using only 2D convolutions.
• Achieved favorable results against state-of-the-art methods on various action recognition datasets.
• Demonstrated efficiency in terms of computational cost and parameters.

Conclusions:

• The proposed STM network offers an efficient and effective solution for action recognition.
• The novel training framework enhances model performance by optimizing feature correlations.
• This approach provides a viable alternative to existing resource-intensive methods.