ShrimpSeg: a local-global structure for the mantis shrimp point cloud segmentation network with contextual reasoning

Area of Science:

• Computer Vision
• Biotechnology
• Aquaculture

Background:

• Accurate measurement of mantis shrimp dimensions is crucial for ideotype selection and phenotyping.
• Manual measurement methods are labor-intensive, costly, and prone to uncertainty.
• Automated organ point cloud segmentation is essential for efficient and accurate phenotypic measurements.

Purpose of the Study:

• To develop an automated framework for organ segmentation of mantis shrimps from multiview stereo (MVS) point clouds.
• To address the gap in research concerning mantis shrimp point cloud segmentation.
• To facilitate precise phenotypic measurements for aquaculture applications.

Main Methods:

• A Transformer-based MVS architecture was employed to generate dense point clouds from calibrated phone images.
• An improved point cloud segmentation method, ShrimpSeg, was developed, utilizing local and global features with contextual information.
• ShrimpSeg was specifically designed for organ segmentation in mantis shrimps.

Main Results:

• The developed framework successfully segmented mantis shrimp organs from MVS point clouds.
• ShrimpSeg achieved a per-class intersection over union (IoU) of 82.4% for organ-level segmentation.
• Experimental results demonstrated that ShrimpSeg outperformed commonly used segmentation methods.

Conclusions:

• The proposed automated framework and ShrimpSeg method are effective for mantis shrimp organ segmentation.
• This work provides a foundation for improving shrimp phenotyping and advancing intelligent aquaculture.
• The developed system shows potential for production-ready applications in aquaculture.