PointFormer: Keypoint-Guided Transformer for Simultaneous Nuclei Segmentation and Classification in Multi-Tissue...

Area of Science:

• Digital Pathology
• Computational Biology
• Medical Image Analysis

Background:

• Accurate nuclei segmentation and classification (NSC) is crucial for digital pathology and precision medicine.
• Existing methods struggle with overlapping nuclei, large intra-class variability, and complex clinical data.
• Transformer-based methods show promise but require further investigation for NSC adaptability.

Purpose of the Study:

• To develop an advanced method for simultaneous nuclei segmentation and classification (NSC) in digital pathology.
• To address challenges like severe nuclei overlap and intra-class variability in clinical histopathological images.
• To improve the accuracy and efficiency of nuclei detection and classification (NDC) for biomarker quantification.

Main Methods:

• Proposed PointFormer, a keypoint-guided tri-decoder Transformer architecture for unified NSC.
• Decoupled NSC into a multi-task learning problem with decoders for nuclei instance, edges, and types.
• Reformulated nuclei detection and classification (NDC) as a semantic keypoint estimation problem with attention-guiding.

Main Results:

• PointFormer demonstrated superior performance over prevalent methods on three diverse datasets.
• Achieved a significant 70.6% bPQ score on the challenging PanNuke dataset.
• The method effectively handles overlapping nuclei and large intra-class variability.

Conclusions:

• PointFormer offers a robust and effective solution for nuclei segmentation and classification in digital pathology.
• The proposed approach enhances the potential for accurate biomarker quantification and precision medicine applications.
• The tri-decoder structure and keypoint-guided strategy significantly improve NSC performance on complex datasets.