A Novel Geometric Framework on Gram Matrix Trajectories for Human Behavior Understanding.

Area of Science:

• Computer Vision
• Geometric Deep Learning
• Human Motion Analysis

Background:

• Analyzing human landmark configurations over time is crucial for understanding actions and emotions.
• Existing methods often lack a robust way to incorporate spatial covariance with shape representation.
• Modeling temporal dynamics on complex manifolds is an ongoing challenge.

Purpose of the Study:

• To propose a novel space-time geometric representation for human landmark configurations.
• To develop tools for comparing and classifying these configurations based on their temporal evolution.
• To improve accuracy in tasks like action and emotion recognition using 3D skeletal and video data.

Main Methods:

• Landmarks are mapped to a Riemannian manifold of positive semidefinite matrices, forming time-parameterized trajectories.
• Geometric and computational tools are derived for rate-invariant analysis and adaptive re-sampling.
• A temporal warping technique provides a geometry-aware dissimilarity measure, integrated into a Support Vector Machine (SVM) classifier.

Main Results:

• The proposed representation naturally incorporates spatial covariance alongside affine-shape information.
• Rate-invariant analysis and adaptive re-sampling tools are developed based on Riemannian geometry.
• Competitive results are demonstrated in action recognition, emotion recognition from 3D skeletal data, and facial expression recognition from videos.

Conclusions:

• The novel space-time geometric representation offers a powerful framework for analyzing human motion.
• The developed geometric tools enable robust and accurate comparison and classification of landmark trajectories.
• The approach shows significant potential for advancing human behavior analysis in computer vision applications.