Compositional action recognition with multi-view feature fusion.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Current action recognition models often process videos as single events.
• Representing activities as verb-noun combinations improves action understanding.
• Limited research exists on cross-view or cross-modality learning for action recognition.

Purpose of the Study:

• To develop a feature fusion framework leveraging complementary information from multiple views.
• To enhance action recognition by integrating appearance features and multi-view data.
• To improve generalization and identification of unseen actions.

Main Methods:

• Proposed a two-step feature fusion framework: appearance feature extraction and multi-view feature fusion.
• Validated the approach on the IKEA ASM and LEMMA action recognition datasets.
• Employed multi-view fusion to exploit complementary information between different video perspectives.

Main Results:

• Multi-view fusion effectively generalizes across appearances and identifies previously unseen actions.
• The proposed method surpasses current state-of-the-art action recognition techniques.
• Achieved an 18.1% improvement in top-1 performance on the IKEA ASM dataset compared to single-view methods.

Conclusions:

• Multi-view feature fusion is a powerful technique for advancing action recognition.
• The framework demonstrates significant improvements in generalization and performance on complex actions.
• This approach offers a promising direction for future research in cross-view and cross-modality learning.