MonSter++: Unified Stereo Matching, Multi-View Stereo, and Real-Time Stereo With Monodepth Priors.

Area of Science:

• Computer Vision
• Geometric Deep Learning

Background:

• Multi-view depth estimation, including stereo matching, faces challenges in ill-posed regions due to limited matching cues.
• Existing methods struggle to reconcile single-view and multi-view depth information effectively.

Purpose of the Study:

• To introduce MonSter++, a novel geometric foundation model for multi-view depth estimation.
• To unify rectified stereo matching and unrectified multi-view stereo by integrating monocular depth priors.
• To enhance depth estimation accuracy, particularly in challenging, ill-posed regions.

Main Methods:

• Developed MonSter++, a dual-branched architecture fusing monocular and multi-view depth.
• Implemented confidence-based guidance for adaptive selection of multi-view cues.
• Utilized iterative mutual enhancement between monocular and multi-view depth predictions.
• Employed a cascaded search and multi-scale depth fusion strategy.

Main Results:

• MonSter++ achieves new state-of-the-art performance on stereo matching and multi-view stereo benchmarks.
• The real-time variant, RT-MonSter++, significantly outperforms previous real-time methods.
• Demonstrated significant improvements across eight benchmarks covering stereo matching, real-time stereo matching, and multi-view stereo.
• Exhibited superior zero-shot generalization capabilities.

Conclusions:

• MonSter++ effectively integrates monocular priors to overcome limitations in multi-view depth estimation.
• The framework demonstrates strong generality and achieves state-of-the-art accuracy and real-time performance.
• The proposed method advances the field by enabling fine-grained, pixel-level geometry reconstruction.