Monocular-to-3D Virtual Try-On With Generative Semantic Articulated Fields.

Area of Science:

• Computer Vision
• Computer Graphics
• Artificial Intelligence

Background:

• Existing 3D virtual try-on methods often require expensive 3D scans or pseudo-depth maps.
• These requirements limit dataset creation and model scalability for virtual try-on applications.

Purpose of the Study:

• To introduce a novel monocular-to-3D virtual try-on network (G3D-VTON) capable of synthesizing multi-view try-on results from single monocular images.
• To develop a 3D-aware conditional Generative Adversarial Network (3D-GAN) that simplifies dataset construction and enhances model scalability by training solely on 2D images.

Main Methods:

• The Generative monocular-to-3D Virtual Try-ON network (G3D-VTON) integrates a 3D-aware conditional Parsing Module (3DPM), a U-Net Refinement Module (URM), and a Flow-based 2D Virtual Try-On Module (FTM).
• The 3DPM generates a 3D representation using conditional generative semantic articulated fields and leverages the 3D SMPL prior via inverse skinning to learn the Signed Distance Function (SDF).
• Deferred pose guidance decouples style and pose conditions for view-controllable generation, while the URM and FTM refine outputs for accurate and realistic details.

Main Results:

• The proposed G3D-VTON effectively generates faithful 3D human appearances wearing desired garments from monocular images.
• The method outperforms existing 3D-GAN and depth-based 3D approaches in generating high-quality virtual try-on results.
• The G3D-VTON achieves superior visual fidelity in both 3D and 2D outputs compared to prior methods.

Conclusions:

• The G3D-VTON offers a scalable and effective solution for monocular-to-3D virtual try-on by utilizing a 3D-aware GAN trained on 2D images.
• The integrated modules successfully address limitations of previous methods, producing more accurate and realistic virtual try-on results.
• This approach simplifies dataset requirements and demonstrates significant advancements in synthesizing multi-view try-on experiences.