Mutual Component Convolutional Neural Networks for Heterogeneous Face Recognition.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Heterogeneous face recognition (HFR) faces challenges due to significant modality differences (e.g., visible vs. near-infrared) and limited training data.
• Existing methods struggle to effectively bridge the gap between diverse facial modalities.

Purpose of the Study:

• To propose a novel modal-invariant deep learning framework, the Mutual Component Convolutional Neural Network (MC-CNN), for robust HFR.
• To simultaneously address the challenges of large modality discrepancy and insufficient training samples in HFR.

Main Methods:

• The MC-CNN integrates Mutual Component Analysis (MCA) as a specialized fully-connected layer within a deep convolutional neural network architecture.
• MCA is employed to extract modal-independent hidden factors from deep features, utilizing a maximum likelihood analytic formulation for updates to prevent overfitting.
• A novel MCA loss function is developed to guide the network towards learning modal-invariant features.

Main Results:

• The proposed MC-CNN significantly outperforms several fine-tuned baseline models in heterogeneous face recognition tasks.
• The framework demonstrates state-of-the-art performance on benchmark datasets including CASIA NIR-VIS 2.0, CUHK NIR-VIS, and IIIT-D Sketch.

Conclusions:

• The MC-CNN framework offers an effective solution for heterogeneous face recognition by learning modal-invariant representations.
• The integration of MCA and a specialized loss function enables superior performance, particularly in scenarios with limited and diverse training data.