Semi-supervised multi-modal medical image segmentation with unified translation

Area of Science:

• Medical image analysis
• Deep learning
• Computer vision

Background:

• Deep learning for medical image segmentation faces challenges with multi-modal data and limited expert annotations.
• Existing semi-supervised methods often work with single modalities and cannot fully utilize complementary information from multiple sources.
• Current semi-supervised multi-modal approaches have rigid structures and require modality-specific training.

Purpose of the Study:

• To propose a novel, flexible semi-supervised method for multi-modal medical image segmentation.
• To leverage complementary information from multi-modal data to improve segmentation performance.
• To address the challenges of insufficient annotations and diverse data representations in semi-supervised multi-modal learning.

Main Methods:

• Developed a flexible method named semi-supervised multi-modal medical image segmentation with unified translation (SMSUT).
• Employed unified translation to extract complementary information and focus on inter-modality disparities and salient features.
• Implemented pixel-level and feature-level constraints to handle annotation scarcity and data diversity.
• Introduced a novel training procedure integrating conditional translation for semi-supervised multi-modal medical image analysis.

Main Results:

• The proposed SMSUT method demonstrated superior performance compared to existing semi-supervised segmentation models on public datasets.
• The model effectively leverages multi-modal information, showing high-performance capabilities.
• The method exhibits remarkable adaptability to varying numbers of modalities in training data.

Conclusions:

• The developed SMSUT method offers a flexible and effective solution for semi-supervised multi-modal medical image segmentation.
• The approach successfully utilizes complementary information from multiple modalities to enhance segmentation accuracy.
• The proposed method shows strong performance and transferability, addressing key challenges in the field.