Face Inverse Rendering via Hierarchical Decoupling.

Area of Science:

• Computer Vision
• Computer Graphics
• Deep Learning

Background:

• Traditional face inverse rendering methods rely on synthetic data or specialized equipment, limiting real-world applicability.
• A significant gap exists between synthetic and real-world data, hindering model generalization.
• The complexity and cost of professional equipment make face inverse rendering inaccessible for general users.

Purpose of the Study:

• To develop a deep learning framework for disentangling face images into albedo, normal, and lighting components from in-the-wild images.
• To overcome the limitations of existing methods regarding data requirements and generalization.
• To enable more accessible and broadly applicable face inverse rendering.

Main Methods:

• A novel deep learning framework is proposed for face inverse rendering.
• A decomposition network employing a hierarchical subdivision strategy is utilized.
• The method takes image pairs captured from arbitrary viewpoints as input.

Main Results:

• The proposed framework successfully disentangles face images into albedo, normal, and lighting components.
• The approach demonstrates robust performance on real-world images, mitigating data preparation challenges.
• Experimental results show superior performance in face relighting compared to state-of-the-art alternatives.

Conclusions:

• The developed deep learning framework offers a practical solution for face inverse rendering using in-the-wild images.
• The method significantly broadens the applicability of face inverse rendering by reducing reliance on synthetic data and professional equipment.
• The approach achieves superior face relighting performance, paving the way for wider adoption.