Self-supervision assisted multimodal remote sensing image classification with coupled self-looping convolution...

Area of Science:

• Remote Sensing
• Geospatial Data Analysis
• Deep Learning

Background:

• Multimodal data fusion is increasingly used in remote sensing for tasks like land cover classification.
• Current deep learning (DL) methods face challenges including forward-only architectures, high labeled data needs, and limited cross-modal interaction.

Purpose of the Study:

• To propose a novel self-supervision oriented method for multimodal remote sensing data fusion.
• To enhance feature extraction and cross-modal learning capabilities.
• To address limitations of existing DL techniques in handling multimodal data.

Main Methods:

• A self-supervised auxiliary task is employed to reconstruct input features from one modality using another's extracted features.
• A bidirectional convolutional framework with self-looping connections is utilized for a self-correcting architecture.
• Shared parameters are incorporated across modality-specific extractors to facilitate cross-modal communication.

Main Results:

• The proposed method achieved high accuracies on three datasets: Houston 2013 (93.08%), Houston 2018 (84.59%), and TU Berlin (73.21%).
• Performance surpassed the state-of-the-art by at least 3.02%, 2.23%, and 2.84% on the respective datasets.
• The approach demonstrated effective cross-modal learning and improved pre-fusion feature representation.

Conclusions:

• The novel self-supervision method effectively fuses multimodal remote sensing data.
• The bidirectional architecture and cross-modal coupling enhance feature representation and model robustness.
• This approach offers a promising solution for supervised and unsupervised multimodal data fusion challenges in remote sensing.