Perceptual Real-Time 2D-to-3D Conversion Using Cue Fusion.

Area of Science:

• Computer Vision
• Perception
• Real-time Systems

Background:

• Traditional depth reconstruction in computer vision often yields numerically accurate but perceptually unconvincing results.
• Existing 2D-to-3D conversion systems lack agreement between luminance and disparity, and struggle with fine details.

Purpose of the Study:

• To develop a real-time system for inferring perceptually plausible binocular disparity from monocular video streams.
• To achieve a realistic depth impression with plausible layout, sharp edges, and fine details.
• To validate the concept of perceptually plausible disparity through comparisons and user studies.

Main Methods:

• Estimating low-resolution disparity and confidence maps using monocular cues in real-time.
• Employing spatially-varying, appearance-dependent, and class-specific disparity priors learned from stereo images.
• Utilizing robust MAP inference on a dense spatio-temporal conditional random field for fusing cues and priors.

Main Results:

• The system generates perceptually plausible disparity maps, offering a similar depth impression to real stereo vision.
• The approach achieves real-time performance with constant-time, parallel per-pixel computations.
• Comparisons with existing 2D-to-3D systems and user studies validate the effectiveness of the proposed method.

Conclusions:

• The proposed system successfully infers perceptually plausible disparity from monocular video in real-time.
• This method provides a compelling alternative to traditional depth reconstruction for enhanced visual experiences.
• The concept of perceptually plausible disparity is validated as a key factor in realistic depth perception.