Visual Prompting Based Incremental Learning for Semantic Segmentation of Multiplex Immuno-Flourescence Microscopy...

Area of Science:

• Medical image analysis
• Computational neuroscience
• Biomedical engineering

Background:

• Deep learning excels at semantic segmentation but requires large datasets.
• Medical image segmentation faces challenges due to limited annotated data.
• Existing methods struggle with the constraints of medical imaging datasets.

Purpose of the Study:

• To develop an efficient framework for fine-tuning multi-class segmentation of multiplex immuno-fluorescence images.
• To address the challenge of limited annotated data in medical image segmentation.
• To improve the accuracy and reduce the annotation effort for medical image segmentation tasks.

Main Methods:

• A modified Swin-UNet architecture was employed for initial global segmentation (pre-training).
• An incremental learning approach was used for refining each class with minimal expert-provided labels.
• Multi-class weights were utilized for initialization, guiding network optimization for specific regions.

Main Results:

• The proposed framework significantly outperforms traditional multi-class segmentation methods.
• The incremental learning approach drastically reduces the amount of required expert labeling.
• Annotation speed is substantially increased, enabling rapid error correction by experts.

Conclusions:

• The developed framework offers an effective solution for semantic segmentation of medical images with limited data.
• Incremental fine-tuning with minimal annotations enhances segmentation accuracy and efficiency.
• This approach holds promise for accelerating research in fields requiring detailed medical image analysis.