CMOS-GAN: Semi-Supervised Generative Adversarial Model for Cross-Modality Face Image Synthesis.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Machine Learning

Background:

• Cross-modality face image synthesis is crucial for applications like face recognition and digital entertainment.
• Traditional methods often require paired training data, which is difficult to obtain.
• Unpaired data is more common, presenting a challenge for existing synthesis models.

Purpose of the Study:

• To propose a novel semi-supervised cross-modality synthesis method (CMOS-GAN) that effectively utilizes both paired and unpaired face images.
• To develop a robust cross-modality synthesis model capable of leveraging diverse data sources.
• To improve face recognition accuracy through enhanced synthetic modalities.

Main Methods:

• Utilized an encoder-decoder generator architecture for synthesizing new modalities.
• Employed a combination of pixel-wise loss, adversarial loss, classification loss, and face feature loss for model training.
• Incorporated a modified triplet loss to preserve discriminative subject features in synthetic images.

Main Results:

• Demonstrated the effectiveness of CMOS-GAN across three cross-modality tasks: NIR-to-VIS, RGB-to-depth, and sketch-to-photo.
• Achieved superior performance compared to state-of-the-art methods.
• Introduced and collected a large-scale RGB-D dataset (VIPL-MumoFace-3K) for RGB-to-depth synthesis.

Conclusions:

• CMOS-GAN offers a robust and effective solution for semi-supervised cross-modality face image synthesis.
• The method successfully leverages both paired and unpaired data, overcoming limitations of conventional approaches.
• The developed technique shows potential for improving face recognition systems and contributes a valuable new dataset to the research community.