Structure-Preserving Binary Representations for RGB-D Action Recognition.

Area of Science:

• Computer Vision
• Machine Learning
• Data Fusion

Background:

• RGB-D data offers rich information for action recognition but fusing it effectively remains challenging.
• Existing methods often struggle to preserve structural information during feature extraction and fusion.
• Gradient field analysis provides a robust way to represent local image structures.

Purpose of the Study:

• To propose a novel binary local representation for effective RGB-D video data fusion.
• To introduce the Local Flux Feature (LFF) for capturing general features from video data.
• To enhance action recognition performance by fusing RGB and depth information using Structure Preserving Projection (SPP).

Main Methods:

• Describing gradient fields of RGB and depth information to acquire general features.
• Developing Local Flux Features (LFF) based on local fluxes (orientation and magnitude) of gradient fields.
• Fusing LFFs from RGB-D channels into Hamming space via Structure Preserving Projection (SPP) with shape constraints and bipartite graph considerations.

Main Results:

• Demonstrated high efficiency of binary codes for data representation.
• Showcased the effectiveness of LFFs fused via SPP on various RGB-D action recognition benchmarks.
• Validated the potential of LFF for general action recognition tasks.

Conclusions:

• The proposed LFF combined with SPP offers an effective approach for RGB-D video data fusion.
• The method successfully preserves data structure and improves action recognition accuracy.
• LFF shows promise for broader applications in computer vision and action recognition.