A Lightweight Neural Network for Monocular View Generation With Occlusion Handling

Area of Science:

• Computer Vision
• Deep Learning
• Computer Graphics

Background:

• Multi-view image formats are increasingly prevalent, driving demand for view synthesis techniques.
• Generating novel views from a single image presents challenges, particularly in handling occluded regions.
• Existing methods often require significant computational resources and parameters.

Purpose of the Study:

• To develop a lightweight neural network architecture for synthesizing novel views from a single image.
• To improve occlusion handling and depth map generation in view synthesis.
• To reduce the number of parameters required for state-of-the-art view synthesis performance.

Main Methods:

• A novel neural network architecture trained on stereo image pairs for view synthesis.
• Disparity estimation combined with an occlusion handling technique.
• Learning and replicating left-right consistency constraints from stereo data for single-image inference.
• Blending disparity-based predictions with direct minimization in occluded regions.

Main Results:

• The proposed network generates left and right views, depth maps, and pixelwise confidence measures from a single input image.
• Achieved state-of-the-art performance, both visually and metrically, on the KITTI dataset.
• Significantly reduced the number of parameters (5-10x fewer, 6.5M total) compared to existing methods.

Conclusions:

• The lightweight neural network effectively synthesizes novel views and depth information from single images.
• The method demonstrates superior performance and efficiency, making it suitable for resource-constrained applications.
• The occlusion handling strategy and learned stereo consistency are key to the model's success.