LeOp-GS: Learned Optimizer With Dynamic Gradient Update for Sparse-View 3DGS.

Area of Science:

• Computer Vision
• Computer Graphics
• Machine Learning

Background:

• 3D Gaussian Splatting (3DGS) excels in novel view synthesis but struggles with sparse-view data, leading to overfitting and poor reconstruction.
• Existing methods lack effective solutions for optimizing 3DGS with limited input views.

Purpose of the Study:

• To develop an innovative approach for optimizing 3D Gaussian Splatting (3DGS) using a learned optimizer, specifically addressing challenges posed by sparse-view inputs.
• To enhance the reconstruction quality and stability of 3DGS models when trained with limited training data.

Main Methods:

• A learning-to-optimize framework is employed, utilizing a multi-layer perceptron (MLP) as a learned optimizer for 3DGS parameters.
• A point-wise position-aware optimizer is introduced to individually update parameters for each 3DGS point based on coordinates and current values.
• A dynamic gradient update strategy involving spatial perturbation and weighted fusion is proposed for training the optimizer.

Main Results:

• The proposed position-aware optimizer effectively imposes constraints, leading to stable convergence and improved parameter solutions in sparse-view scenarios.
• The learned optimizer successfully mitigates overfitting and enhances reconstruction quality for 3DGS with sparse training views.
• Experimental results demonstrate state-of-the-art performance across multiple datasets, validating the method's efficacy.

Conclusions:

• The developed learned optimizer offers a robust solution for optimizing 3D Gaussian Splatting (3DGS) with sparse-view inputs.
• This approach significantly improves the stability and quality of 3DGS reconstructions in challenging data-limited scenarios.
• The method represents a significant advancement in leveraging learned optimizers for 3D computer vision tasks.