Discriminative Relational Representation Learning for RGB-D Action Recognition.

Area of Science:

• Computer Vision
• Machine Learning
• Human-Computer Interaction

Background:

• Human action recognition from video is crucial for applications like surveillance and robotics.
• Existing methods often struggle with fusing heterogeneous data like RGB and depth information.
• Robustness to missing modalities (RGB or depth) remains a challenge in multimodal action recognition.

Purpose of the Study:

• To propose a novel discriminative relational feature learning method for RGB-D human action recognition.
• To effectively fuse heterogeneous RGB and depth data by learning shared semantic features.
• To enhance classification accuracy and robustness, especially when one modality is absent.

Main Methods:

• A discriminative relational feature learning approach is proposed.
• Factorizes feature matrices for each modality (RGB, depth) to learn shared features.
• Employs a hinge loss within a maximum margin framework for supervised factorization and classification.
• Utilizes a coordinate descent algorithm for optimization.

Main Results:

• The method learns extremely low-dimensional features with superior discriminative power.
• Achieves state-of-the-art performance on public RGB-D action datasets.
• Demonstrates high performance robustness when either RGB or depth modality is missing during training or testing.

Conclusions:

• The proposed method effectively fuses multimodal RGB-D data for human action recognition.
• It learns discriminative and low-dimensional features, outperforming existing approaches.
• The method shows significant robustness to missing modalities, making it practical for real-world scenarios.