Adaptive Light Estimation using Dynamic Filtering for Diverse Lighting Conditions.

Area of Science:

• Computer Vision
• Computer Graphics
• Artificial Intelligence

Background:

• High dynamic range (HDR) panoramic environment maps are crucial for realistic illumination of virtual objects in augmented and mixed reality (AR/MR).
• Consumer-level devices often lack the capability to capture full 360° surroundings for HDR environment map generation.
• Existing methods struggle with dynamic lighting variations and limited field-of-view (FOV) inputs.

Purpose of the Study:

• To develop a novel light estimation method for predicting 360° HDR environment maps from single, limited FOV photographs.
• To introduce a dynamic convolutional neural network (DLNet) that adaptively learns lighting cues from input images.
• To enable high-fidelity AR/MR applications by providing accurate and consistent environmental lighting information.

Main Methods:

• Introduction of the Dynamic Lighting network (DLNet), a convolutional neural network.
• Development of novel Spherical Multi-Scale Dynamic (SMD) convolutional modules for spherical domain feature decoding.
• Utilizing DLNet with data augmentations for FOV, exposure multiplier, and color temperature variations.

Main Results:

• DLNet dynamically generates sample-specific convolution filters, adapting to input photograph lighting cues.
• The proposed SMD modules effectively predict 360° environment maps by decoding spherical domain features.
• The model demonstrates robust light estimation accuracy across varying FOVs, exposure multipliers, and color temperatures.
• Maintained lighting consistency across different exposure multipliers and color temperatures, outperforming prior fixed-filter methods.

Conclusions:

• The developed light estimation method accurately predicts 360° HDR environment maps from limited FOV images.
• DLNet offers a significant advancement for AR/MR applications by enabling coherent illumination of virtual objects.
• This approach supports high-fidelity AR/MR across diverse environmental lighting conditions and consumer-level device sensors.