Do It Once: Concatenating the Image Pair for a Single Pass Feature Extraction in Stereo Depth Sensing

Area of Science:

• Computer Vision
• Machine Learning
• Robotics

Background:

• Stereo depth sensing is crucial for real-time applications but faces speed limitations on resource-constrained platforms.
• Current acceleration methods often involve complex redesigns or accuracy compromises.
• A key inefficiency lies in redundant feature extraction using separate passes for left and right images.

Purpose of the Study:

• To accelerate stereo depth sensing inference speed without compromising accuracy.
• To address the redundancy in feature extraction within existing stereo pipelines.
• To propose a universally applicable enhancement for stereo matching models.

Main Methods:

• Concatenated left and right images into a single tensor for batched feature extraction.
• Utilized the same network weights for both images in a single forward pass.
• Preserved the original network architecture, avoiding redesigns.

Main Results:

• Reduced feature extraction time by up to 48.4%.
• Achieved overall inference rate acceleration of 10%–39% on Nvidia V100 and up to 28.4% on edge devices.
• Maintained original accuracy with only a moderate increase in memory consumption.

Conclusions:

• The proposed method offers a plug-and-play solution for enhancing stereo depth sensing speed.
• It effectively overcomes the inference speed bottleneck for real-time applications.
• This technique is compatible with both existing and new stereo matching models.