X-ModalNet: A semi-supervised deep cross-modal network for classification of remote sensing data.

Area of Science:

• Remote Sensing
• Computer Vision
• Machine Learning

Background:

• Large-scale remote sensing data (multispectral imagery - MSI, synthetic aperture radar - SAR) enables global urban scene analysis.
• Pixel-wise classification using MSI/SAR is limited by noisy data, poor discriminative information, and scarce annotated training images.

Purpose of the Study:

• To develop a novel cross-modal deep-learning framework for semi-supervised transfer learning in remote sensing.
• To enhance pixel-wise classification accuracy by transferring knowledge from limited hyperspectral images (HSI) to large-scale MSI or SAR data.

Main Methods:

• Proposed X-ModalNet framework with three modules: self-adversarial, interactive learning, and label propagation.
• Learning to transfer discriminative information from HSI to MSI/SAR data.
• Utilizing semi-supervised learning via label propagation on an updatable graph constructed from high-level features.

Main Results:

• X-ModalNet demonstrated significant improvement in pixel-wise classification accuracy on HSI-MSI and HSI-SAR datasets.
• The framework achieved superior performance compared to several state-of-the-art methods.
• Effective knowledge transfer and generalization were observed through label propagation.

Conclusions:

• X-ModalNet provides an effective solution for semi-supervised cross-modality learning in remote sensing.
• The proposed method addresses the challenge of limited annotated data in remote sensing classification tasks.
• The framework shows strong potential for improving material identification in urban scenes using diverse remote sensing data.