3D Scene Creation and Rendering via Rough Meshes: A Lighting Transfer Avenue

Area of Science:

• Computer Graphics
• Computer Vision
• Artificial Intelligence

Background:

• Existing 3D reconstruction techniques yield rough 3D models (R3DMs), leading to low-quality renderings in Physically-Based Rendering (PBR).
• Neural Fields, like NeRFs, offer photo-realistic renderings but fail to capture lighting details crucial for PBR pipelines, especially with local shadows.

Purpose of the Study:

• To develop a method for seamlessly integrating Neural Fields Rendering (NFR) and PBR pipelines.
• To enhance the realism of 3D scenes by improving lighting and surface quality in rendered models.

Main Methods:

• Proposed LighTNet, a lighting transfer network designed to bridge NFR and PBR.
• Implemented a simplified image composition model and addressed uneven surfaces from R3DMs.
• Introduced perceptual constraints and a novel Lab angle loss for enhanced lighting contrast.

Main Results:

• LighTNet effectively synthesizes impressive lighting effects.
• The network improves the integration of NFR and PBR, enhancing overall scene quality.
• Demonstrated superiority in synthesizing lighting compared to existing methods.

Conclusions:

• LighTNet successfully bridges the gap between NFR and PBR, enabling mutual benefits.
• The proposed method shows significant promise for advancing NFR in practical 3D modeling workflows.
• LighTNet enhances the visual quality of 3D scenes by improving lighting and surface rendering.