Characterizing humans on Riemannian manifolds

Area of Science:

• Computer Vision
• Machine Learning
• Differential Geometry

Background:

• Assessing human behavioral traits in surveillance relies heavily on head and body orientation.
• Individuals are often represented by few, noisy pixels, complicating characterization in surveillance.
• Existing methods for pedestrian detection use covariances on Riemannian manifolds for binary classification.

Purpose of the Study:

• To extend covariance-based descriptors for multiclassification of human orientation from limited pixel data.
• To propose a novel descriptor, the weighted array of covariances, for analyzing tiny image representations.
• To develop a differential geometry approach for applying machine learning to covariance data on Riemannian manifolds.

Main Methods:

• Developed a novel descriptor: the weighted array of covariances.
• Utilized a differential geometry approach to project covariances onto a tangent space.
• Applied the Campbell-Baker-Hausdorff expansion for efficient approximation of geodesic distances on the manifold.
• Integrated standard machine learning techniques within the tangent space framework.

Main Results:

• Demonstrated the effectiveness of the weighted array of covariances for multiclassification tasks.
• Achieved convincing results on multiple benchmark and newly proposed datasets.
• Showcased the framework's ability to handle characterization from tiny, noisy image representations.

Conclusions:

• The proposed computational framework offers a robust solution for analyzing human orientation and behavior from limited surveillance data.
• The weighted array of covariances descriptor, combined with differential geometry, significantly advances the state-of-the-art in analyzing complex visual patterns.
• This methodology provides a powerful tool for enhancing surveillance applications through improved behavioral assessment.