Learning-based, automatic 2D-to-3D image and video conversion.

Area of Science:

• Computer Vision
• Machine Learning
• 3D Graphics

Background:

• The availability of 3D content significantly lags behind 2D content.
• Existing 2D-to-3D conversion methods are either manual (costly, time-consuming) or automatic but lack quality due to reliance on flawed assumptions.
• There is a need for efficient and high-quality 2D-to-3D conversion techniques.

Purpose of the Study:

• To propose and evaluate new learning-based methods for automatic 2D-to-3D image and video conversion.
• To demonstrate the feasibility of using existing 3D content repositories for depth map generation.
• To offer computationally efficient alternatives to current conversion methods.

Main Methods:

• Developed two learning-based methods for 2D-to-3D conversion.
• Method 1: Pixel-wise regression mapping local image/video attributes (color, position, motion) to scene depth.
• Method 2: Nearest-neighbor regression for global depth map estimation using a repository of 3D image data (image+depth pairs or stereopairs).

Main Results:

• Demonstrated the effectiveness and computational efficiency of both proposed methods on various 2D images.
• Achieved promising results in 2D-to-3D conversion, indicating the utility of 3D content repositories.
• The methods show potential for extension to video by ensuring temporal continuity of depth maps.

Conclusions:

• Learning-based approaches from 3D content repositories offer a viable solution for 2D-to-3D conversion.
• The proposed methods provide an efficient and effective alternative to existing techniques.
• Further development can extend these methods for high-quality 3D video generation.