Spatio-Frequency Decoupled Weak-Supervision for Face Reconstruction.

Area of Science:

• Computer Vision
• 3D Reconstruction
• Deep Learning

Background:

• Current 3D face reconstruction methods struggle with texture detail, limiting photorealistic rendering.
• Weak supervision solely in the spatial domain hinders the harmony between reconstructed shape and texture.

Purpose of the Study:

• To propose a spatio-frequency decoupled weak-supervision method for improved 3D face reconstruction.
• To achieve photorealistic facial image rendering by integrating spatial and frequency domain losses.
• To generate high-quality 3D face shapes and detailed textures using minimal supervision.

Main Methods:

• Implemented image-level and perceptual-level spatial domain losses.
• Separated frequency domain information from input and rendered images to create frequency-based loss.
• Devised a spectrum-wise weighted Wing loss for balanced attention across different spectrums.
• Utilized landmark labels as the sole supervision signal.

Main Results:

• The proposed method successfully generates detailed textures and high-quality shapes.
• Experimental results demonstrate superior performance compared to state-of-the-art methods.
• Qualitative and quantitative comparisons confirm the effectiveness of the spatio-frequency decoupled approach.

Conclusions:

• Spatio-frequency decoupled weak-supervision enables harmonious learning for 3D face reconstruction.
• The method achieves photorealistic effects with high fidelity using only landmark supervision.
• This approach significantly advances the state-of-the-art in 3D face reconstruction.