A dual-task segmentation network based on multi-head hierarchical attention for 3D plant point cloud.

Area of Science:

• Computer Vision
• Plant Science
• Agricultural Technology

Background:

• Automated high-throughput plant phenotyping requires accurate semantic and instance segmentation of botanical structures.
• Existing methods often rely on empirical thresholds and lack unified frameworks for dual-level segmentation.

Purpose of the Study:

• To develop an integrated methodology for non-destructive, high-throughput plant phenotyping.
• To address limitations in current plant structure segmentation approaches by unifying semantic and instance segmentation.

Main Methods:

• A Dual-Task Segmentation Network (DSN) was developed using 2D images of Caladium bicolor, reconstructed into 3D point clouds.
• The DSN incorporates a multi-head hierarchical attention mechanism and Multi-Value Conditional Random Field (MV-CRF) for joint optimization.
• A dual-branch architecture predicts semantic class and embeds points for instance clustering.

Main Results:

• The DSN achieved 99.16% macro-averaged precision and 93.64% Intersection over Union for semantic segmentation.
• Instance segmentation metrics reached 87.94%, outperforming nine benchmark architectures.
• Ablation studies confirmed the effectiveness of the network's design choices.

Conclusions:

• The proposed DSN framework offers a superior, unified approach to semantic and instance segmentation for 3D plant point clouds.
• This methodology significantly advances automated plant phenotyping capabilities, enabling more precise non-destructive analysis.