Heterogeneous Face Interpretable Disentangled Representation for Joint Face Recognition and Synthesis.

Area of Science:

• Computer Science
• Biometrics
• Artificial Intelligence

Background:

• Heterogeneous face analysis is crucial for real-world biometric security but faces challenges due to modality discrepancies.
• Existing methods for heterogeneous face analysis often lack interpretability or introduce synthesis bias.

Purpose of the Study:

• To develop a method for learning interpretable representations of heterogeneous faces.
• To simultaneously perform face recognition and synthesis tasks across different modalities.
• To address the limitations of current approaches in interpretability and synthesis bias.

Main Methods:

• Proposed the heterogeneous face interpretable disentangled representation (HFIDR) for explicit interpretation of face representation dimensions.
• Developed a multimodality extension (M-HFIDR) for handling multiple face modalities.
• Constructed a large-scale face sketch dataset to evaluate generalization capabilities.

Main Results:

• The proposed HFIDR method enables explicit interpretation of face representation dimensions.
• Achieved effective cross-modality face recognition by extracting latent identity information.
• Demonstrated successful cross-modality face synthesis by converting modality factors.
• Experimental results on multiple databases confirmed the method's effectiveness.

Conclusions:

• The developed HFIDR and M-HFIDR methods offer an interpretable and effective solution for heterogeneous face analysis.
• The approach successfully bridges the modality gap, improving both recognition and synthesis.
• The study contributes a novel method and dataset for advancing heterogeneous face recognition research.