Paced-curriculum distillation with prediction and label uncertainty for image segmentation

Area of Science:

• Artificial Intelligence
• Machine Learning
• Medical Image Analysis

Background:

• Curriculum learning and self-paced learning train models on data samples of increasing difficulty.
• Both methods require effective data sample difficulty scoring, which remains an area of active research.
• Distillation transfers knowledge from a teacher to a student network using various samples.

Purpose of the Study:

• To develop an uncertainty-based paced curriculum learning strategy within self-distillation for medical image segmentation.
• To improve model generalization and robustness by guiding student networks with an efficient curriculum strategy.
• To introduce a novel paced-curriculum distillation (P-CD) method.

Main Methods:

• P-CD fuses prediction uncertainty from a teacher model and annotation boundary uncertainty derived from spatially varying label smoothing.
• Uncertainty quantification was employed to guide the curriculum learning process.
• The method's robustness was evaluated against various image perturbations and corruptions.

Main Results:

• The P-CD technique demonstrated significantly improved segmentation performance on breast ultrasound and robot-assisted surgical datasets.
• The method achieved enhanced robustness against image perturbations and corruptions.
• Validation on two distinct medical imaging datasets confirmed the effectiveness of P-CD.

Conclusions:

• Paced-curriculum distillation (P-CD) enhances performance, generalization, and robustness, particularly across dataset shifts.
• The proposed method effectively addresses limitations associated with hyper-parameter tuning in traditional curriculum learning.
• P-CD offers a promising approach for advancing medical image segmentation.