PlantSAM: An object detection-driven segmentation pipeline for herbarium specimens

Area of Science:

• Botany
• Computer Science
• Digital Imaging

Background:

• Herbarium image classification using deep learning faces challenges due to heterogeneous backgrounds.
• Background noise and artifacts can mislead models and reduce classification accuracy.

Purpose of the Study:

• To develop an automated segmentation pipeline, PlantSAM, for enhancing herbarium image analysis.
• To improve the accuracy of deep learning-based classification of botanical traits from herbarium images.

Main Methods:

• PlantSAM integrates YOLOv10 for object detection and Segment Anything Model (SAM2) for segmentation.
• Both YOLOv10 and SAM2 were fine-tuned on herbarium images, with YOLOv10 providing bounding box prompts for SAM2.
• Performance was evaluated using Intersection over Union (IoU) and Sørensen-Dice coefficient.

Main Results:

• PlantSAM achieved state-of-the-art segmentation performance with an IoU of 0.94 and a Sørensen-Dice coefficient of 0.97.
• Integrating segmented images into classification models improved performance across five botanical traits.
• Accuracy gains reached up to 4.36% and F1 score improvements reached 4.15%.

Conclusions:

• Background removal is crucial for improving herbarium image analysis.
• Automated segmentation enhances deep learning models' ability to focus on plant structures, leading to better classification outcomes.