Masks-to-Skeleton: Multi-View Mask-Based Tree Skeleton Extraction with 3D Gaussian Splatting

Area of Science:

• Computer Vision
• Computational Geometry
• Plant Science

Background:

• Reconstructing 3D tree skeletons from multi-view images is difficult.
• Existing methods using 3D point clouds struggle with thin branches and low-texture contrast, leading to fragmented data and broken connectivity.

Purpose of the Study:

• To develop a robust method for accurate 3D tree skeleton reconstruction.
• To overcome limitations of point-cloud-based approaches by leveraging image segmentation masks.

Main Methods:

• A mask-guided graph optimization framework is proposed.
• The 3D skeleton is modeled as a graph, optimized using 3D Gaussian splatting (3DGS) for silhouette rendering and a minimum spanning tree (MST) algorithm for regularization.

Main Results:

• The new method directly estimates 3D skeletons from multi-view segmentation masks.
• Demonstrates consistent improvements in completeness and structural accuracy compared to existing methods.

Conclusions:

• The mask-guided graph optimization framework offers a more reliable approach to 3D tree skeleton reconstruction.
• This method enhances structural accuracy and completeness, particularly for challenging plant structures.