Towards Lightweight Pixel-Wise Hallucination for Heterogeneous Face Recognition.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Biometrics

Background:

• Heterogeneous Face Recognition (HFR) faces challenges due to modality discrepancies between different types of face images.
• Cross-spectral face hallucination is a common approach to address HFR, but often suffers from shape misalignment.
• Existing methods for HFR often rely on complex architectures to overcome shape misalignment issues.

Purpose of the Study:

• To propose a simple and effective method, Shape Alignment FacE (SAFE), to mitigate shape misalignment in cross-spectral face hallucination for HFR.
• To develop a lightweight generator for spectrum translation that focuses solely on spectral characteristics after shape alignment.
• To enhance the practicality of HFR systems by reducing computational complexity and enabling training on limited data.

Main Methods:

• Utilizes a 3D face model to align the shape of a given face image to its paired heterogeneous counterpart.
• Employs a lightweight generator for spectrum translation, supervised at the pixel level on the aligned image pairs.
• Introduces a probabilistic pixel-wise loss to handle residual discrepancies and employs spectrum optimal transport in a shape-irrelevant latent space to mitigate misalignment effects.

Main Results:

• SAFE achieves superior performance in both qualitative synthesis and quantitative recognition across six datasets.
• The method employs a significantly more lightweight generator, boasting 128x fewer parameters and 63x fewer FLOPs than state-of-the-art methods, with low latency.
• SAFE demonstrates effective performance on low-shot datasets, such as Oulu-CASIA NIR-VIS, which is a novel capability in HFR.

Conclusions:

• The proposed SAFE method effectively addresses shape misalignment in cross-spectral face hallucination for HFR.
• SAFE offers a practical solution with a highly efficient generator and the ability to train on small datasets, pushing the boundaries of HFR.
• The method's advantages in efficiency and data requirements make it suitable for real-world HFR applications.