CloseUpShot: Close-Up Novel View Synthesis From Sparse-Views via Point-Conditioned Diffusion Model.

Area of Science:

• Computer Vision
• 3D Scene Reconstruction
• Novel View Synthesis

Background:

• Reconstructing 3D scenes and synthesizing novel views from sparse inputs is challenging.
• Video diffusion models show promise for sparse-view reconstruction but struggle with close-up details due to limited input.
• Existing methods fail to capture fine-grained details in close-up scenarios.

Purpose of the Study:

• To develop a diffusion-based framework for close-up novel view synthesis from sparse inputs.
• To enhance the quality and completeness of conditioning images for video diffusion models.
• To provide consistent geometric context for sparse conditioning inputs.

Main Methods:

• Proposed CloseUpShot, a point-conditioned video diffusion framework.
• Introduced hierarchical warping and occlusion-aware noise suppression to improve conditioning images.
• Implemented global structure guidance using a dense fused point cloud for geometric consistency.

Main Results:

• CloseUpShot outperforms existing approaches in close-up novel view synthesis.
• Demonstrated effectiveness on multiple datasets.
• Validated the improvements from hierarchical warping, occlusion-aware noise suppression, and global structure guidance.

Conclusions:

• The proposed framework effectively addresses the challenges of close-up novel view synthesis from sparse inputs.
• Hierarchical warping, occlusion-aware noise suppression, and global structure guidance are crucial for enhancing reconstruction quality.
• CloseUpShot represents a significant advancement in sparse-view 3D scene reconstruction and novel view synthesis.