S³R: Shape and Semantics-Based Selective Regularization for Explainable Continual Segmentation Across Multiple Sites.

Area of Science:

• Medical Imaging
• Computer Vision
• Machine Learning

Background:

• Continual learning in medical image segmentation is crucial for multi-site data streams due to storage and privacy constraints.
• Existing methods face challenges in retaining complex shape and semantic information from previous sites and lack explainability.
• There is a need for methods that can effectively learn sequentially from diverse data sources without forgetting past knowledge.

Purpose of the Study:

• To propose a novel method for explainable cross-site continual segmentation that preserves both shape and semantic knowledge from previously learned sites.
• To address the limitations of weak memorizability and poor explainability in current continual learning approaches for medical imaging.
• To develop a technique that allows medical image segmentation models to adapt to new data sites while retaining expertise from prior sites.

Main Methods:

• Introduced a Shape and Semantics-based Selective Regularization (S3R) method for continual segmentation.
• Developed a selective regularization scheme that penalizes parameter changes based on Joint Shape and Semantics-based Importance (JSSI) weights.
• Proposed an Importance Activation Mapping (IAM) technique for visualizing memorized content and interpreting memory consolidation.

Main Results:

• The S3R method effectively maintained both shape and semantic knowledge from previously learned sites during cross-site continual learning.
• The Importance Activation Mapping (IAM) provided clear visualization and interpretation of the memory consolidation process.
• Evaluations on prostate and optic cup/disc segmentation tasks demonstrated superior performance compared to existing methods in reducing model forgetting.

Conclusions:

• The proposed S3R method offers a robust solution for explainable cross-site continual medical image segmentation.
• The approach significantly reduces model forgetting and enhances the explainability of memory consolidation.
• This work advances the field by enabling more effective and interpretable continual learning in multi-site medical imaging applications.