SALENet: Structure-Aware Lighting Estimations from a Single Image for Indoor Environments

Area of Science:

• Computer Vision
• Computer Graphics
• Machine Learning

Background:

• High Dynamic Range (HDR) lighting is crucial for realistic augmented and mixed reality (AR/MR) experiences.
• Acquiring precise HDR environment maps is currently expensive and impractical with standard equipment.

Purpose of the Study:

• To develop a cost-effective method for estimating global lighting conditions from single images for AR/MR applications.
• To mitigate the need for resource-intensive HDR map acquisition techniques.

Main Methods:

• Introduced SALENet, a novel deep network utilizing a hierarchical transformer architecture.
• Implemented a cross-attention mechanism between different resolution lighting source representations.
• Employed a structure-based contrastive learning method for improved lighting prediction accuracy.

Main Results:

• SALENet effectively estimates global lighting conditions from a single image.
• The hierarchical transformer and contrastive learning synergy significantly enhanced lighting prediction accuracy.
• Optimized spatial distribution of lighting sources for improved consistency.

Conclusions:

• SALENet enables high-fidelity augmented and mixed reality with cost-effective, realistic lighting effects.
• The proposed method addresses the limitations of traditional HDR environment map acquisition.
• This research facilitates wider adoption of immersive AR/MR technologies.