MonoRelief V2: Leveraging Real Data for High-Fidelity Monocular Relief Recovery.

Area of Science:

• Computer Vision
• 3D Reconstruction
• Machine Learning

Background:

• Recovering 2.5D reliefs from single images is challenging due to complex material and illumination variations.
• Previous models like MonoRelief V1 were limited by training solely on synthetic data, impacting real-world performance.
• There is a need for robust models that can handle diverse real-world conditions.

Purpose of the Study:

• To introduce MonoRelief V2, an improved end-to-end model for direct 2.5D relief recovery from single images.
• To enhance model robustness, accuracy, and efficiency by incorporating real-world data into the training process.
• To demonstrate state-of-the-art performance in depth and normal prediction for 2.5D relief reconstruction.

Main Methods:

• Developed MonoRelief V2, an end-to-end deep learning model.
• Generated a large-scale pseudo-real dataset (15,000 images) using text-to-image models and fused depth/normal predictions for pseudo-labels.
• Constructed a small-scale real-world dataset (800 samples) using multi-view reconstruction and refinement.
• Progressively trained MonoRelief V2 on both pseudo-real and real-world datasets.

Main Results:

• MonoRelief V2 achieves state-of-the-art performance in both depth and normal predictions.
• The model demonstrates improved robustness and accuracy compared to its predecessor.
• Experiments validate the effectiveness of training with a combination of pseudo-real and real-world data.

Conclusions:

• MonoRelief V2 offers a significant advancement in single-image 2.5D relief recovery.
• The integration of real data and advanced data generation techniques enhances model generalization.
• The model shows strong potential for various downstream applications in computer vision and graphics.