VAT: Visibility Aware Transformer for Fine-Grained Clothed Human Reconstruction

Area of Science:

• Computer Vision
• 3D Reconstruction
• Deep Learning

Background:

• Reconstructing detailed 3D clothed humans from limited images is challenging.
• Existing methods struggle with fine-grained geometric accuracy.

Purpose of the Study:

• To develop a robust framework for accurate 3D clothed human reconstruction from sparse views.
• To improve fine-grained geometric detail learning by leveraging global geometry.

Main Methods:

• A deep cooperating two-level global-to-fine reconstruction framework.
• A novel visibility-aware Transformer (VAT) integrating global and fine-grained levels.
• A multi-scale feature extractor (MSUNet) for aligned feature extraction.

Main Results:

• The framework effectively reconstructs 3D clothed humans with accurate fine-grained details.
• Demonstrated significant advantages over state-of-the-art methods in performance and generalization.
• The visibility-aware attention mechanism efficiently fuses multi-view and multi-scale features.

Conclusions:

• The proposed framework enables high-fidelity 3D clothed human reconstruction.
• Deep cooperation between global and fine-grained levels is crucial for detailed geometry.
• The method shows strong potential for various computer vision applications.