Purely Attention Based Local Feature Integration for Video Classification.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Convolutional Neural Networks (CNNs) and Recurrent Neural Networks (RNNs) are commonly used for video classification.
• Existing methods often struggle to effectively capture temporal patterns and integrate local features.

Purpose of the Study:

• To investigate a purely attention-based approach for local feature integration in video classification.
• To propose and evaluate a novel model, Pyramid x Pyramid Attention Clusters (PPAC), for improved video classification accuracy.

Main Methods:

• Proposed Basic Attention Clusters (BAC) with a shifting operation for initial feature integration.
• Introduced channel pyramid attention for fine-grained channel dimension integration.
• Developed temporal pyramid attention to preserve sequential relationships in feature sequences.
• Combined both pyramid attention mechanisms into the final PPAC model.

Main Results:

• BAC demonstrated excellent performance on multiple datasets.
• PPAC effectively models both channel and temporal dimensions for richer feature integration.
• The proposed PPAC model achieved competitive and consistent results across seven real-world video classification datasets.

Conclusions:

• The PPAC framework offers a superior approach to local feature integration for video classification.
• The model consistently outperforms existing methods by effectively capturing hierarchical feature representations and temporal dynamics.