Jointly Learning Heterogeneous Features for RGB-D Activity Recognition.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Activity recognition using RGB-D data is crucial for human-computer interaction.
• Heterogeneous features from different sensors (e.g., RGB, depth) present challenges in learning unified representations.
• Existing methods often struggle to effectively exploit shared underlying structures across diverse feature modalities.

Purpose of the Study:

• To develop a novel joint learning model for heterogeneous feature learning in RGB-D activity recognition.
• To simultaneously explore shared and feature-specific components within RGB and depth data.
• To enable effective cross-dataset feature fusion through transfer learning of intermediate transforms.

Main Methods:

• A unified framework for heterogeneous multi-task learning is proposed.
• The model jointly mines shared subspaces and quantifies feature-specific components.
• A three-step iterative optimization algorithm is employed for efficient training, followed by a simple inference model.

Main Results:

• The proposed method demonstrates significant efficacy across four benchmark activity datasets.
• The joint learning approach effectively exploits latent shared features and enhances feature fusion.
• The transfer of feature-specific intermediate transforms improves cross-dataset learning capabilities.

Conclusions:

• The joint learning model offers a robust solution for heterogeneous feature learning in RGB-D activity recognition.
• The approach successfully addresses the challenge of fusing information from different sensor modalities.
• The study contributes a new RGB-D dataset for human-object interaction, advancing activity recognition research.