Single-view 3D reconstruction via dual attention.

Area of Science:

• Computer Vision
• Artificial Intelligence
• 3D Graphics

Background:

• Simultaneous construction of global context and local fine-grained information is crucial for single-view 3D reconstruction.
• Existing transformer networks face challenges in maintaining linear complexity while capturing both global and local features.

Purpose of the Study:

• To propose a novel network, R3Davit, for effective single-view 3D reconstruction.
• To leverage spatial and channel dimension attention for comprehensive feature extraction.
• To improve inference speed and non-linear learning capabilities in 3D reconstruction.

Main Methods:

• The R3Davit network employs an encoder based on the Davit backbone, utilizing spatial and channel attention.
• The decoder incorporates a nonlinear reinforcement block, a selective state space model block, and an up-sampling Residual Block, avoiding self-attention layers.
• The selective state space model block replaces self-attention, maintaining linear complexity.

Main Results:

• R3Davit outperforms state-of-the-art methods on the ShapeNet and ShapeNetChairRFC datasets, achieving 1% higher IOU and 2% higher F1 scores.
• Experiments on the Pix3d dataset demonstrate the method's robustness on real-world data.
• The proposed decoder design effectively learns features while maintaining inference speed.

Conclusions:

• R3Davit offers a significant advancement in single-view 3D reconstruction by effectively balancing global context and local details.
• The network's architecture, particularly the Davit backbone and specialized decoder, provides superior performance and robustness.
• The proposed approach sets a new benchmark for single-view 3D reconstruction accuracy and efficiency.