Weakly supervised segmentation with cross-modality equivariant constraints.

Area of Science:

• Computer Vision
• Machine Learning
• Medical Image Analysis

Background:

• Weakly supervised learning offers an alternative to large labeled datasets for semantic segmentation.
• Current methods using class activation maps (CAMs) from image-level annotations produce suboptimal pixel-level labels.
• Enhancing CAMs is crucial for improving weakly supervised semantic segmentation.

Purpose of the Study:

• To develop a novel weakly supervised learning strategy for semantic segmentation.
• To leverage self-supervision in multi-modal imaging to enhance class activation maps (CAMs).
• To address the limitations of existing CAM-based methods by improving label quality.

Main Methods:

• A novel learning strategy utilizing self-supervision in multi-modal image scenarios.
• Exploiting commonalities between image modalities as a self-supervisory signal to correct CAM inconsistencies.
• Integrating a novel loss function with within-modality and cross-modality equivariant terms.
• Employing KL-divergence on class prediction distributions to facilitate inter-modality information exchange.

Main Results:

• Significantly enhanced original CAMs through the proposed self-supervision strategy.
• Demonstrated superior performance on multi-modal BraTS and prostate DECATHLON datasets.
• Outperformed relevant recent literature under identical learning conditions.

Conclusions:

• The proposed method effectively enhances CAMs for weakly supervised semantic segmentation in multi-modal settings.
• Leveraging cross-modal self-supervision and equivariant constraints improves segmentation accuracy.
• This approach offers a promising direction for reducing reliance on extensive pixel-level annotations.