Related Experiment Video
Updated: Jun 29, 2025
12:08
From Voxels to Knowledge: A Practical Guide to the Segmentation of Complex Electron Microscopy 3D-Data
Published on: August 13, 2014
24.6K
Enhanced Nuclei Segmentation and Classification via Category Descriptors in the SAM Model
Miguel Luna1, Philip Chikontwe1, Sang Hyun Park1,2
1Department of Robotics and Mechatronics Engineering, Daegu Gyeongbuk Institute of Science and Technology (DGIST), Daegu 42988, Republic of Korea.
Bioengineering (Basel, Switzerland)
|March 27, 2024
Summary
This study enhances nuclei segmentation and classification in histopathology images using the Segment Anything Model (SAM). It improves detection of rare nuclei types by aligning image features, boosting F1 scores by up to 12%.
Area of Science:
- Digital Pathology
- Computational Biology
- Medical Image Analysis
Background:
- Nuclei segmentation and classification in H&E histopathology images face challenges due to the long-tailed distribution of cell types.
- Foundation models for image segmentation, such as the Segment Anything Model (SAM), show promise for improving the detection of rare nuclei.
Purpose of the Study:
- To adapt the Segment Anything Model (SAM) for accurate nuclei segmentation and classification in histopathology images.
- To address the domain gap between natural scene images and histopathology images for improved model generalization.
- To enhance the detection of rare nuclei types.
Main Methods:
- Utilized category descriptors to prompt the SAM model for nuclei segmentation and classification.
- Implemented feature alignment in low-level space to bridge the domain gap while preserving SAM's high-level representations.
- Validated the approach on the Lizard dataset.
Main Results:
- Achieved automatic nuclei segmentation and classification, particularly improving the detection of rare nuclei types.
- Demonstrated a significant improvement in F1 score by up to 12% for rare nuclei detection.
- Maintained compatibility with manual point prompts for interactive refinement without additional training.
Conclusions:
- The proposed method effectively leverages SAM for nuclei analysis in histopathology.
- The approach successfully enhances the identification of rare cell types, crucial for accurate diagnoses.
- The model offers a flexible and efficient tool for digital pathology applications.
