Panoramic Stereo Imaging of a Bionic Compound-Eye Based on Binocular Vision

Area of Science:

• Computer Vision
• Virtual Reality Technology
• Optical Engineering

Background:

• The virtual reality (VR) industry faces a significant bottleneck due to the limited availability of high-quality video resources.
• Capturing high-definition panoramic video with accurate depth information and enabling real-time stereo display are critical challenges for immersive VR experiences.

Purpose of the Study:

• To propose an optical optimization design for panoramic imaging utilizing binocular stereo vision.
• To develop a real-time panoramic stereo imaging system by integrating advanced image processing algorithms.

Main Methods:

• An ultra-high precision panoramic camera calibration platform was constructed using a theodolite angle compensation function.
• The projection matrix for adjacent cameras was determined using binocular stereo vision principles.
• A real-time registration algorithm for multi-detector mosaic images, incorporating the Lucas-Kanade optical flow method and image segmentation, was developed for stereo matching and depth estimation.

Main Results:

• The developed system achieved stereo matching in 30 milliseconds.
• A registration accuracy of 0.1 pixels was attained.
• The system produced depth maps with clearer edge information and demonstrated robustness across various lighting conditions.

Conclusions:

• The proposed optical design and real-time processing algorithms effectively address the limitations in panoramic stereo imaging for VR.
• The developed system provides a viable solution for capturing high-definition panoramic video with depth information, crucial for advancing VR applications.