Sparse Canonical Temporal Alignment with Deep Tensor Decomposition for Action Recognition.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Action recognition faces challenges with sub-action, multi-subject, and multi-modality recognition due to intra-class sample diversity.
• Existing temporal alignment methods can lead to misalignment and reduced performance with dense frames.

Purpose of the Study:

• To address limitations in action recognition by reducing intra-class sample diversity.
• To propose novel methods for effective feature extraction and temporal alignment in action recognition.

Main Methods:

• Sparse Canonical Temporal Alignment (SCTA) for selecting and aligning key frames to reduce diversity.
• Deep Non-negative Tensor Factorization (DNTF) to extract features by modeling action sequences as spatiotemporal tensors.
• A two-layer tensor decomposition using Non-negative Tensor Factorization (NTF) and Tensor-Train (TT) for efficiency.

Main Results:

• The integrated SCTA and DNTF framework effectively solves sub-action, multi-subject, and multi-modality action recognition problems.
• Experimental results on synthetic and real-world datasets (MSRDailyActivity3D, MSRActionPairs) demonstrate superior accuracy compared to existing methods.

Conclusions:

• The proposed SCTA and DNTF framework offers a robust solution for complex action recognition tasks.
• This approach significantly improves recognition accuracy by effectively handling sample diversity and extracting discriminative features.