A subspace model-based approach to face relighting under unknown lighting and poses

Area of Science:

• Computer Vision
• Computer Graphics
• Image Processing

Background:

• Face relighting aims to synthesize realistic facial images under novel lighting conditions.
• Existing methods often rely on 3D face shape models, limiting their applicability.
• Accurate estimation of pose, reflectance, and lighting is crucial for high-quality relighting.

Purpose of the Study:

• To develop a new face relighting approach that bypasses the need for 3D face shape reconstruction.
• To enable the synthesis of face images under arbitrary lighting conditions from minimal input.
• To preserve non-Lambertian skin properties and facial details during relighting.

Main Methods:

• A pose- and pixel-dependent subspace model of reflectance functions was trained using large face databases (e.g., CMU PIE, Yale).
• Joint estimation of facial pose, reflectance functions, and lighting from a single input image.
• Subspace model enables estimation without explicit 3D face shape recovery.

Main Results:

• The proposed method successfully estimates pose, reflectance, and lighting from single images.
• Synthesized face images exhibit high subjective and objective quality under new lighting.
• Preservation of non-Lambertian reflectance, facial hair, and realistic shadow reproduction was achieved.

Conclusions:

• The novel approach offers a robust and efficient solution for face relighting.
• Eliminating the 3D shape reconstruction step simplifies the process and broadens applicability.
• The method outperforms existing techniques in terms of visual quality and accuracy.