Difference-complementary Learning and Label Reassignment for Multimodal Semi-Supervised Semantic Segmentation of...

Area of Science:

• Remote Sensing
• Computer Vision
• Machine Learning

Background:

• Multimodal remote sensing image fusion, particularly optical and Synthetic Aperture Radar (SAR) data, is crucial for land cover semantic segmentation.
• Challenges include differing data characteristics, noise interference, and limited labeled samples in practical applications.
• Existing semi-supervised methods struggle with compromised pseudo-label quality in complex regions, hindering performance.

Purpose of the Study:

• To develop an effective semi-supervised semantic segmentation method for multimodal remote sensing images.
• To address the challenges of data heterogeneity, noise, and limited labels.
• To improve the accuracy and robustness of land cover classification.

Main Methods:

• Introduced the Difference-complementary Learning and Label Reassignment (DLLR) network.
• Employed asymmetric masking and difference-guided complementary learning for mutual modality learning.
• Implemented a multi-level label reassignment strategy using optimal transport for precise pseudo-label annotation.
• Utilized multimodal consistency cross pseudo-supervision for enhanced pseudo-label utilization.

Main Results:

• The DLLR network demonstrated superior performance in multimodal semantic segmentation.
• Experimental results on WHU-OPT-SAR and EErDS-OPT-SAR datasets confirmed its effectiveness.
• The proposed methods significantly improved the accuracy of land cover classification compared to existing deep networks.

Conclusions:

• The DLLR framework effectively handles multimodal data fusion challenges for semantic segmentation.
• The label reassignment and complementary learning strategies enhance pseudo-label quality and model performance.
• DLLR offers a promising solution for semi-supervised semantic segmentation in remote sensing with limited labeled data.