Transformer Based Binocular Disparity Prediction with Occlusion Predict and Novel Full Connection Layers.

Area of Science:

• Computer Vision
• Deep Learning
• Artificial Intelligence

Background:

• Convolutional Neural Networks (CNNs) for depth estimation face limitations in disparity range, occlusion handling, and global context perception.
• Traditional methods struggle with acquiring accurate disparities beyond a predefined range and ensuring matching uniqueness.

Purpose of the Study:

• To propose a Transformer-based disparity prediction algorithm that addresses the shortcomings of CNNs in depth estimation.
• To enhance feature extraction, disparity matching, and occlusion prediction for improved depth accuracy.

Main Methods:

• Utilized a Swin-SPP module for feature extraction based on Swin Transformer.
• Developed a Transformer disparity matching network employing self-attention and cross-attention mechanisms.
• Integrated an occlusion prediction sub-network and a double skip connection fully connected layer to improve training stability and inference accuracy.

Main Results:

• Achieved an EPE (Absolute Error) of 0.57 on KITTI 2012 and 0.61 on KITTI 2015.
• Obtained a 3PE (Percentage Error > 3px) of 1.74% on KITTI 2012 and 1.56% on KITTI 2015.
• Demonstrated efficient inference time (0.46s) with a low parameter count (2.6M).

Conclusions:

• The proposed Transformer-based model significantly outperforms existing algorithms in depth estimation accuracy and efficiency.
• The novel architecture effectively handles limitations related to disparity range, occlusion, and global context.
• The algorithm shows great advantages in various evaluation metrics for stereo depth estimation.