TKRL: Targeted Knowledge Rectification Learning Against Teacher-Originated Defects in Domain Continual Segmentation.

Area of Science:

• Computer Vision
• Machine Learning
• Deep Learning

Background:

• Catastrophic forgetting is a major challenge in domain continual segmentation.
• Existing knowledge distillation methods propagate defects from older models, exacerbating forgetting.
• Teacher-originated defects like knowledge gaps and biases hinder model performance.

Purpose of the Study:

• To propose a novel framework, Targeted Knowledge Rectification Learning (TKRL), to address defects in older models during continual segmentation.
• To mitigate catastrophic forgetting by rectifying teacher-originated knowledge gaps and biases.
• To enhance the performance of domain continual segmentation models.

Main Methods:

• TKRL employs Probe-augmented Class Distillation to identify and transfer underrepresented features, bridging knowledge gaps.
• TKRL utilizes a Variance-guided Masked Autoencoder to reconstruct high-uncertainty patches, correcting inherited biases.
• The framework focuses on probing and correcting defects within older teacher models.

Main Results:

• TKRL effectively rectifies knowledge gaps and biases inherited from older models.
• The proposed framework significantly mitigates catastrophic forgetting in domain continual segmentation.
• Experimental results demonstrate enhanced performance in continual segmentation tasks.

Conclusions:

• TKRL offers an effective solution for mitigating catastrophic forgetting in domain continual segmentation.
• Correcting teacher-originated defects is crucial for improving knowledge transfer and model performance.
• The proposed method advances the field of continual learning for segmentation tasks.