Gait Recognition and Understanding Based on Hierarchical Temporal Memory Using 3D Gait Semantic Folding.

Area of Science:

• Computer Vision
• Biometrics
• Machine Learning

Background:

• Gait recognition systems struggle with unstructured image/video data, facing challenges from multi-views, occlusion, clothing, and carrying objects.
• Existing methods are often sensitive to variations in appearance and viewing conditions, limiting real-world applicability.

Purpose of the Study:

• To develop a robust gait recognition system using realistic 3D human structural data and a novel sequential pattern learning framework.
• To address the performance degradation issues caused by unstructured data and environmental variations in current gait recognition technologies.

Main Methods:

• Proposed a 2D to 3D human body pose and shape parameter estimation method using instance-level body parsing and virtual dressing.
• Introduced gait semantic folding to encode body parameters into sparse 2D matrices, creating structural gait semantic images.
• Utilized a Hierarchical Temporal Memory (HTM) network with a top-down attention mechanism to generate sequence-level gait sparse distribution representations (SL-GSDRs) for time-based recognition.

Main Results:

• The proposed method significantly improves accuracy and robustness in gait recognition across multiple datasets (CMU MoBo, CASIA B, TUM-IITKGP, KY4D).
• The top-down attention mechanism effectively refines SL-GSDRs, mitigating issues related to multi-views and other challenging conditions.
• Generated structured gait semantic images provide valuable input for visual cognition tasks.

Conclusions:

• The developed framework enhances gait recognition performance in realistic, challenging scenarios.
• The integration of 3D structural data, HTM, and attention mechanisms offers a promising direction for advanced biometrics.
• The system provides structured visual representations of gait, aiding both recognition and cognitive understanding.