DiffusionLight-Turbo: Accelerated Light Probes for Free via Single-Pass Chrome Ball Inpainting.

Area of Science:

• Computer Vision
• Artificial Intelligence
• Computer Graphics

Background:

• Estimating lighting from low-dynamic-range (LDR) images is crucial for realistic rendering.
• Existing methods struggle with generalization due to limited high-dynamic-range (HDR) panorama datasets.
• Diffusion models offer potential but face challenges with content consistency and HDR output.

Purpose of the Study:

• To introduce an efficient and effective technique for estimating lighting from a single LDR image.
• To overcome the generalization limitations of previous lighting estimation methods.
• To significantly reduce the computational time for lighting estimation.

Main Methods:

• Reframed lighting estimation as a chrome ball inpainting problem using Stable Diffusion XL.
• Introduced DiffusionLight, employing iterative inpainting and median aggregation for stable lighting priors.
• Developed DiffusionLight-Turbo with a Turbo LoRA and LoRA swapping for a 60x runtime reduction (approx. 30 seconds).
• Fine-tuned an Exposure LoRA for generating HDR light probes from multiple LDR images.

Main Results:

• DiffusionLight-Turbo achieves a 60x speedup over DiffusionLight, reducing estimation time to ~30 seconds with minimal quality degradation.
• The method produces convincing light estimates across diverse settings.
• Demonstrated superior generalization capabilities in in-the-wild scenarios compared to existing techniques.

Conclusions:

• DiffusionLight-Turbo offers a highly efficient and effective solution for single-image lighting estimation.
• The technique leverages diffusion models for robust performance and generalization.
• The code is publicly available, facilitating further research and application.