Unifying Flow, Stereo and Depth Estimation.

Area of Science:

• Computer Vision
• Machine Learning
• Robotics

Background:

• Traditional 3D perception relies on specialized models for optical flow, stereo matching, and depth estimation.
• Integrating these tasks often requires complex, multi-model pipelines.

Purpose of the Study:

• To develop a single, unified model for optical flow, stereo matching, and depth estimation.
• To leverage Transformer-based cross-attention for improved feature representation and cross-task transfer.

Main Methods:

• Formulated optical flow, stereo matching, and depth estimation as a unified dense correspondence matching problem.
• Utilized a Transformer with cross-attention to learn discriminative features and enable cross-view interactions.
• Employed a single model architecture with shared parameters for all three tasks.

Main Results:

• The unified model demonstrated superior performance on the Sintel dataset compared to RAFT.
• Achieved state-of-the-art or competitive results on 10 diverse optical flow, stereo, and depth estimation datasets.
• Showcased significant improvements in feature quality through cross-attention mechanisms.

Conclusions:

• A unified Transformer-based approach offers a simpler and more efficient solution for multiple 3D perception tasks.
• Cross-attention effectively integrates information across views, enhancing feature representations.
• The proposed method enables effective cross-task knowledge transfer, improving overall performance and efficiency.