Dynamic sparse point voxel transformer for 3D point cloud instance segmentation of dormant apple trees

Area of Science:

• Computer Vision
• Agricultural Technology
• Plant Science

Background:

• Accurate 3D tree architecture characterization is vital for fruit crop breeding and orchard robotics.
• Current 3D instance segmentation methods risk losing geometric detail due to coordinate quantization.

Purpose of the Study:

• To develop an efficient and accurate 3D instance segmentation model for high-resolution point clouds of dormant apple trees.
• To overcome limitations of existing methods by preserving fine-grained geometric features.

Main Methods:

• Developed the Dynamic Sparse Point-Voxel Transformer (DSPVFormer) model.
• Employed a hybrid architecture to dynamically map and aggregate raw point features into sparse voxel embeddings.
• Applied the model to high-resolution point clouds of dormant apple trees.

Main Results:

• DSPVFormer achieved statistically significant improvements in instance segmentation metrics compared to baseline models.
• The model demonstrated enhanced accuracy in downstream phenotyping tasks, including branch counting and pruning map generation.
• Results highlight the importance of prioritizing phenotyping-specific metrics for high-throughput phenotyping.

Conclusions:

• DSPVFormer offers efficient and accurate 3D instance segmentation for tree crops, benefiting plant phenotyping and robotic applications.
• The model's ability to capture fine geometric details is crucial for detailed tree analysis.
• Prioritizing task-specific evaluation metrics is essential for realizing the full potential of automated phenotyping systems.