Rethinking exemplars for continual semantic segmentation in endoscopy scenes: Entropy-based mini-batch pseudo-replay.

Area of Science:

• Medical Imaging
• Computer Vision
• Artificial Intelligence

Background:

• Deep learning (DL) models for endoscopic image analysis face catastrophic forgetting, degrading performance on older classes when new data is introduced.
• Data privacy and storage limitations hinder the retraining of DL models with historical endoscopic data.
• Continual learning (CL) is crucial for updating endoscopic image segmentation models without performance loss.

Purpose of the Study:

• To develop a continual learning methodology for endoscopic image segmentation that overcomes catastrophic forgetting.
• To propose a framework, Endoscopy Continual Semantic Segmentation (EndoCSS), that avoids exemplar data storage and privacy issues.
• To enhance the robustness and adaptability of DL models in dynamic endoscopic environments.

Main Methods:

• Introduced the Endoscopy Continual Semantic Segmentation (EndoCSS) framework.
• Implemented a mini-batch pseudo-replay (MB-PR) mechanism using a generative model to create pseudo-replay images, bypassing privacy and storage constraints.
• Incorporated a self-adaptive noisy cross-entropy (SAN-CE) loss function to improve model fitting and training robustness.

Main Results:

• The EndoCSS framework effectively mitigated catastrophic forgetting in endoscopic image segmentation tasks.
• The MB-PR strategy successfully corrected model deviations caused by differing data volumes between replay and current training sets.
• Extensive experiments on public datasets demonstrated robust performance improvements for class increment scenarios.

Conclusions:

• The proposed EndoCSS framework offers a viable solution for continual learning in endoscopic image segmentation.
• The method shows significant potential for real-world deployment in streaming learning scenarios for medical diagnostics.
• The approach effectively addresses catastrophic forgetting without compromising data privacy or requiring extensive storage.