PNProRL: Self-Supervised Neural Relighting via Photometric Perception and Progressive Optimization

Area of Science:

• Computer Vision
• Computer Graphics
• Artificial Intelligence

Background:

• Portrait relighting is crucial for photography, film, and AR applications.
• Current methods require expensive paired data (e.g., online appearance lighting and texture) or synthetic data, limiting scalability.
• Accurately modeling the interplay between physics-guided rendering, neural rendering, and real-world conditions remains a significant challenge.

Purpose of the Study:

• To propose a novel multi-stage self-supervised portrait relighting framework.
• To overcome the limitations of existing methods by removing the need for paired data.
• To adapt to diverse lighting conditions and improve the photorealism and quality of relit portraits.

Main Methods:

• A multi-stage self-supervised framework that progressively refines intrinsic scene properties using a simple-to-complex training strategy.
• A novel pre-training method employing diverse shading-based masking for self-reconstruction to enhance perception of lighting variations.
• Two perceptual modules leveraging the linear superposition of light to bridge physics-guided and neural rendering, aligning results with real-world observations.

Main Results:

• The proposed framework achieves state-of-the-art performance in portrait relighting.
• Demonstrates superior photorealism, synthesis quality, and identity preservation compared to recent methods.
• Effectively adapts to various lighting conditions without requiring paired data.

Conclusions:

• The developed unified framework offers a practical paradigm for high-fidelity portrait relighting.
• The self-supervised approach significantly enhances scalability and adaptability.
• The method successfully narrows the gap between physics-guided and neural rendering for improved real-world alignment.