Unsupervised deep learning for depth estimation with offset pixels.

Area of Science:

• Computer Vision
• Machine Learning
• Sensor Technology

Background:

• Offset Pixel Aperture (OPA) cameras offer single-shot disparity estimation.
• Traditional correspondence matching methods have limitations for OPA disparity estimation.

Purpose of the Study:

• To enhance disparity estimation accuracy in OPA images using a data-oriented approach.
• To address challenges in training deep learning models with OPA's small baseline.

Main Methods:

• Implemented unsupervised deep learning for disparity estimation in OPA images.
• Developed a modified training strategy using displaced camera images to overcome vanishing gradients.
• Evaluated performance against existing single-shot and unsupervised methods.

Main Results:

• Achieved significant performance improvements in disparity estimation compared to prior methods.
• Demonstrated accurate disparity maps with a novel training approach, resolving issues with small baselines.
• The system processes 8 frames per second on an Nvidia 1080 GPU for 1024x512 OPA images.

Conclusions:

• Combining OPA cameras with deep learning creates a compact, accurate depth sensor.
• The method shows utility with real-world, low-quality images and sensors.
• The approach is applicable to small-baseline stereo for short-range depth and multi-baseline stereo for extended depth range.