Recursive Training Strategy for a Deep Learning Network for Segmentation of Pathology Nuclei With Incomplete...

Area of Science:

• Computational pathology
• Medical image analysis
• Artificial intelligence in oncology

Background:

• Accurate nuclei detection and segmentation are crucial for cancer diagnosis and research.
• Incomplete annotations in pathological images pose a significant challenge for training deep learning models.
• Existing methods struggle to effectively leverage limited annotated data for robust nuclei analysis.

Purpose of the Study:

• To develop and validate a recursive training strategy for deep learning-based nuclei detection and segmentation using incomplete annotations.
• To enhance the performance of U-Net models in analyzing H&E stained breast cancer histopathology images.
• To demonstrate the efficacy of the proposed method in improving both nuclei detection sensitivity and segmentation accuracy.

Main Methods:

• A recursive training strategy was developed to iteratively retrain a U-Net model.
• Positive and negative training samples were generated using annotated cells and non-cellular regions, respectively.
• Semi-automated methods were employed for sample selection and quality control during recursive training.
• The strategy involved initial U-Net training, inference, selection of high-quality segmented objects, and retraining with augmented data.

Main Results:

• The recursive training method significantly improved nuclei detection sensitivity from 85.3% to 90.3% (P < 0.05).
• Nuclei segmentation performance showed significant improvement, with average Dice coefficient increasing from 0.780 to 0.831 and Jaccard index from 0.697 to 0.750 (P < 0.05).
• The method effectively enlarged the annotated dataset by incorporating high-quality segmented objects.

Conclusions:

• The proposed recursive training strategy is effective in overcoming challenges posed by incomplete annotations in pathological images.
• This approach substantially enhances the performance of deep learning models for nuclei detection and segmentation in breast cancer.
• The recursive method offers a valuable tool for improving automated analysis of histopathology slides, aiding in more accurate cancer diagnosis and research.