Look More Into Occlusion: Realistic Face Frontalization and Recognition With BoostGAN

Area of Science:

• Computer Vision
• Artificial Intelligence
• Biometrics

Background:

• Face recognition is significantly impacted by factors like occlusion and large pose variations, leading to accuracy degradation.
• Generative Adversarial Networks (GANs) show promise in addressing pose and occlusion issues, but synthesizing and recognizing occluded profile faces remains challenging.
• Existing GAN-based methods focus on either pose-invariant features or face completion, often neglecting identity preservation in occluded regions.

Purpose of the Study:

• To develop an efficient and effective solution for synthesizing and recognizing faces with large pose variations and simultaneous occlusions.
• To address the understudied problem of synthesizing and recognizing occluded yet profile faces.
• To improve the robustness of face recognition systems against combined pose and occlusion challenges.

Main Methods:

• Proposed a boosting Generative Adversarial Network (BoostGAN) for occluded and profile face frontalization, deocclusion, and recognition.
• Incorporated knowledge boosting by feeding multiple partially occluded face images to enhance identity and texture information.
• Designed a novel aggregation structure with a deep encoder-decoder network for coarse synthesis and a boosting network for fine face generation.

Main Results:

• BoostGAN generates photorealistic images of occluded and profile faces.
• Demonstrated powerful recognition performance on benchmark datasets with regular and irregular occlusions.
• Outperformed state-of-the-art GAN models in recognizing occluded but profile faces in both controlled and uncontrolled environments.

Conclusions:

• The proposed BoostGAN effectively synthesizes and recognizes faces under combined pose variations and occlusions.
• This work represents the first attempt to solve face synthesis and recognition under both poses and occlusions simultaneously.
• The method offers a significant advancement in robust face recognition technology for real-world applications.