Deep Heterogeneous Hashing for Face Video Retrieval.

Area of Science:

• Computer Vision
• Machine Learning
• Information Retrieval

Background:

• Face image and video retrieval is crucial for applications like surveillance and content management.
• Existing methods struggle with heterogeneous data spaces (Euclidean for images, Riemannian for videos) and handcrafted features.
• Hashing techniques are vital for efficient large-scale retrieval but face challenges with complex data representations.

Purpose of the Study:

• To develop an end-to-end Deep Heterogeneous Hashing (DHH) method for unified binary code learning for face images and videos.
• To address the challenge of matching heterogeneous data spaces by projecting Riemannian manifold data into Euclidean space.
• To improve the performance of video retrieval using face image queries by integrating feature learning, video modeling, and hashing.

Main Methods:

• An end-to-end Deep Heterogeneous Hashing (DHH) framework integrating image feature learning, video modeling, and heterogeneous hashing.
• Utilizing covariance matrices residing on a Riemannian manifold to model face videos.
• Employing Riemannian kernel mapping to project data into Euclidean space for common Hamming space embedding, considering intra-space discriminability and inter-space compatibility.
• Deriving the gradient of the kernel mapping via structured matrix backpropagation for network optimization.

Main Results:

• The DHH method successfully learns unified binary codes for both face images and videos.
• Experiments on three challenging datasets demonstrate competitive performance compared to existing hashing methods.
• The proposed method effectively handles the heterogeneous spaces matching problem inherent in face video retrieval.

Conclusions:

• Deep Heterogeneous Hashing (DHH) provides an effective solution for retrieving face videos using image queries.
• The integration of manifold learning and deep learning offers a promising direction for heterogeneous hashing tasks.
• The method achieves state-of-the-art or competitive performance, highlighting its practical applicability.