UNI-HoverNet: nuclei segmentation and classification across diverse tissue sections based on the UNI foundation...

Area of Science:

• Computational pathology
• Artificial intelligence in medicine
• Deep learning for medical imaging

Background:

• Accurate segmentation and classification of cell nuclei are crucial for histopathological analysis.
• Existing models may struggle with diverse tissue types and subtle morphological variations.
• The integration of foundation models offers potential for enhanced feature extraction.

Purpose of the Study:

• To introduce UNI-HoverNet, a novel deep learning model for precise nuclei segmentation and classification.
• To leverage the strengths of foundation models and convolutional neural networks for improved performance.
• To enhance feature fusion and utilization for more accurate nuclei detection.

Main Methods:

• Integration of the UNI foundation model into the HoverNet encoder architecture.
• Combination of Convolutional Neural Networks (CNNs) for local features and Vision Transformers for global context.
• Enhancement of the HoverNet decoder with Squeeze-and-Excitation (SE) modules and skip connections for multi-scale feature fusion.

Main Results:

• UNI-HoverNet achieved an average multi-class Panoptic Quality (mPQ) of 0.4733 on the PanNuke dataset.
• The model attained an average F1-score of approximately 0.53 across five nuclei types.
• Significant improvements in Panoptic Quality (PQ) were observed for inflammatory (approx. 3%), connective (approx. 4%), and dead nuclei (approx. 11%) compared to HoverNet.

Conclusions:

• UNI-HoverNet demonstrates superior performance in nuclei segmentation and classification across diverse histopathological tissues.
• The model's architecture effectively combines local and global feature extraction for enhanced accuracy.
• Improved nuclei analysis supports pathologists in understanding tumor progression and identifying therapeutic strategies.