Fine-grained multiclass nuclei segmentation with molecular empowered all-in-SAM model.

Area of Science:

• Computational pathology
• Biomedical image analysis
• Artificial intelligence in medicine

Background:

• Vision foundation models (VFMs), like Segment Anything Model (SAM), are advancing computational pathology.
• Current VFMs struggle with fine-grained semantic segmentation for specific cell types.
• Nuclei segmentation is crucial for accurate pathology image analysis.

Purpose of the Study:

• To introduce the molecular empowered all-in-SAM model for enhanced computational pathology.
• To leverage VFMs for improved nuclei and cell segmentation.
• To address the limitations of general VFMs in fine-grained semantic segmentation.

Main Methods:

• Developed a full-stack approach integrating molecular empowered learning, SAM adapter, and molecular oriented corrective learning.
• Enabled annotation-engaging lay annotators to reduce pixel-level annotation needs.
• Adapted SAM for specific semantics and refined segmentation accuracy.

Main Results:

• The all-in-SAM model significantly improved cell classification performance on diverse datasets.
• Demonstrated robust performance even with varying annotation quality.
• Validated through experiments on both in-house and public datasets.

Conclusions:

• The molecular empowered all-in-SAM model reduces annotator workload and enhances pathology image analysis.
• This approach increases the accessibility of precise biomedical image analysis, especially in resource-limited settings.
• Advances medical diagnostics through automated pathology image analysis.