MAIR++: Improving Multi-View Attention Inverse Rendering With Implicit Lighting Representation.

Area of Science:

• Computer Vision
• Computer Graphics
• Machine Learning

Background:

• Scene-level inverse rendering aims to decompose scenes into geometry, materials (SVBRDF), and lighting from images.
• Existing multi-view methods like MAIR improved upon single-view approaches but had limitations in lighting representation and feature extraction.
• A lack of suitable datasets hindered progress in high-quality, multi-view scene-level inverse rendering.

Purpose of the Study:

• To address the limitations of existing multi-view inverse rendering frameworks, specifically MAIR.
• To introduce an improved framework, MAIR++, capable of more realistic rendering and material editing.
• To enhance the extraction of intricate multi-view relationships for superior scene understanding.

Main Methods:

• Developed MAIR++, an extended version of the Multi-view Attention Inverse Rendering (MAIR) framework.
• Introduced an implicit lighting representation for accurate and realistic rendering.
• Designed a directional attention-based multi-view aggregation network for improved feature extraction.

Main Results:

• MAIR++ significantly outperforms the original MAIR framework and single-view methods in scene-level inverse rendering.
• The implicit lighting representation enables more realistic image rendering and facilitates applications like material editing.
• The directional attention mechanism effectively captures complex inter-view relationships, leading to richer feature extraction.

Conclusions:

• MAIR++ represents a significant advancement in multi-view scene-level inverse rendering.
• The proposed methods overcome key limitations of prior work, enabling more accurate and versatile scene decomposition.
• MAIR++ demonstrates robust performance, even on novel, real-world scenes, highlighting its practical applicability.