Privacy-Preserving Synthetic Continual Semantic Segmentation for Robotic Surgery

Area of Science:

• Robotics
• Computer Vision
• Machine Learning

Background:

• Deep Neural Networks (DNNs) for semantic segmentation improve robotic surgery precision.
• DNNs suffer from catastrophic forgetting in continual learning, especially with data scarcity and privacy concerns.
• Existing methods struggle with incremental learning of new surgical instruments without compromising performance on old ones.

Purpose of the Study:

• To develop a privacy-preserving synthetic continual semantic segmentation framework for robot-assisted surgery.
• To address catastrophic forgetting in DNNs when learning new surgical instruments incrementally.
• To enable robust semantic segmentation without revealing sensitive patient data.

Main Methods:

• A novel framework blending open-source old instrument foregrounds with synthesized backgrounds and new instrument foregrounds with augmented real backgrounds.
• Implementation of overlapping class-aware temperature normalization (CAT) to balance logit distillation.
• Introduction of multi-scale shifted-feature distillation (SD) to preserve long and short-range spatial relationships.

Main Results:

• The proposed framework effectively mitigates catastrophic forgetting in semantic segmentation for surgical instruments.
• Privacy is preserved by utilizing synthetic data, avoiding the need for real patient data release.
• The framework demonstrates effectiveness on the EndoVis 2017 and 2018 instrument segmentation dataset in a generalized continual learning setting.

Conclusions:

• The developed synthetic continual semantic segmentation framework offers a viable solution for privacy-preserving incremental learning in robotic surgery.
• The combination of CAT and SD techniques enhances feature distillation and maintains performance on previously learned tasks.
• This approach advances the development of more adaptable and robust AI systems for surgical applications.