Snapshot 3D Gaussian Splatting for Miniature Scenes

Area of Science:

• Computer Vision
• 3D Reconstruction
• Optical Engineering

Background:

• Miniature scenes are intricate and challenging to reconstruct in 3D.
• 3D digitalization of miniatures is desirable for various applications.
• Existing methods struggle with the complexity and scale of miniature scenes.

Purpose of the Study:

• To present a novel snapshot imaging technique for 3D reconstruction of miniature scenes.
• To design a catadioptric imaging system capable of capturing full-surround views in a single shot.
• To enable 3D digitalization of dynamic miniature scenes.

Main Methods:

• Designed a catadioptric imaging system with multiple planar mirror pairs and a single camera.
• Derived mirror parameters by solving a viewpoint mapping problem for optimal configuration.
• Utilized 3D Gaussian Splatting (3DGS) with a visual hull-derived depth constraint for reconstruction.
• Built a custom imaging system with eight pairs of mirrors on nested pyramid surfaces.

Main Results:

• Successfully captured surrounding multi-view images of miniature scenes in a single shot.
• Achieved superior 3D reconstruction and novel view synthesis compared to state-of-the-art sparse-view and calibration-free 3DGS methods.
• Demonstrated the ability to reconstruct and synthesize novel views of dynamic miniature scenes, including a moving insect.

Conclusions:

• The proposed snapshot imaging technique effectively reconstructs 3D surrounding views of miniature scenes.
• The custom catadioptric system and 3DGS integration overcome challenges in sparse-view reconstruction.
• This method offers a viable solution for the 3D digitalization of intricate, dynamic miniature environments.