SMFormer: Empowering Self-Supervised Stereo Matching via Foundation Models and Data Augmentation

Area of Science:

• Computer Vision
• Machine Learning

Background:

• Self-supervised stereo matching methods often fail due to the photometric consistency assumption, which is violated by real-world disturbances.
• This leads to inaccurate supervisory signals and a performance gap compared to supervised approaches.

Purpose of the Study:

• To develop a robust self-supervised stereo matching framework (SMFormer) that overcomes the limitations of photometric consistency.
• To improve accuracy and compete with supervised methods in stereo matching tasks.

Main Methods:

• Integrated a Vision Foundation Model (VFM) with a Feature Pyramid Network (FPN) for robust feature representation.
• Developed a data augmentation mechanism to enhance robustness against transformations and enforce feature and output consistency.
• Utilized VFM-guided self-supervision and explicit consistency regularization.

Main Results:

• SMFormer achieved state-of-the-art (SOTA) performance among self-supervised methods on multiple benchmarks.
• The framework demonstrated performance comparable to supervised methods.
• Outperformed some SOTA supervised methods on the challenging Booster benchmark.

Conclusions:

• SMFormer offers a more reliable self-supervised approach to stereo matching by integrating VFM and advanced data augmentation.
• The proposed method significantly advances the field of self-supervised computer vision, particularly in disparity estimation.