GPS-Gaussian+: Generalizable Pixel-wise 3D Gaussian Splatting for Real-Time Human-Scene Rendering from Sparse Views

Area of Science:

• Computer Vision
• Computer Graphics
• Machine Learning

Background:

• Differentiable rendering, including Gaussian Splatting and neural implicit methods, shows promise for free-viewpoint video synthesis.
• Current methods often require per-subject optimization, hindering real-time performance in interactive applications.

Purpose of the Study:

• To develop a generalizable Gaussian Splatting approach for high-resolution image rendering from sparse camera views.
• To enable instant novel view synthesis without per-subject optimization or fine-tuning.

Main Methods:

• Introduced Gaussian parameter maps defined on source views and regressed Gaussian properties directly.
• Trained a Gaussian parameter regression module with a depth estimation module to map 2D to 3D.
• Implemented a fully differentiable framework with depth and/or rendering supervision.
• Incorporated a regularization term and epipolar attention for geometry consistency, especially without depth supervision.

Main Results:

• The proposed method achieves high-resolution image rendering from sparse views.
• Demonstrated superior performance compared to state-of-the-art methods in experiments.
• Achieved significantly faster rendering speeds, suitable for real-time applications.

Conclusions:

• The generalizable Gaussian Splatting approach effectively synthesizes novel views from sparse data.
• The method offers a practical solution for real-time rendering in interactive scenarios.
• The framework shows robustness and efficiency in various experimental settings.