Relationship Learning From Multisource Images via Spatial-Spectral Perception Network

Area of Science:

• Remote Sensing
• Computer Vision
• Machine Learning

Background:

• Multisource remote sensing enables comprehensive Earth observation.
• Deep convolutional neural networks (CNNs) show promise in classifying multisource data by integrating spatial-spectral information.
• Existing methods struggle to fully extract spatial distribution and spectral relationships, limiting classification performance.

Purpose of the Study:

• To propose a novel spatial-spectral perception network (S2PNet) for improved multisource remote sensing data classification.
• To effectively extract and integrate complementary information from diverse data sources.
• To address limitations in extracting spatial and spectral feature relationships.

Main Methods:

• Developed a spatial perception network to model spatial distribution from high-resolution imagery.
• Developed a spectral perception network to extract spectral relationships from spectral imagery.
• Utilized a memory unit for successive feature storage and cross-information perception between data sources.
• Introduced distance loss, reconstruction loss, and cross-entropy loss for feature integrity and class discrimination.

Main Results:

• The proposed S2PNet demonstrated superior performance in multisource data classification across multiple datasets.
• Achieved average improvements in overall accuracy of +0.77%, +5.62%, +1.58%, and +1.79% compared to existing classifiers.
• Effectively extracted and leveraged cross-information between different remote sensing data sources.

Conclusions:

• S2PNet offers a significant advancement in multisource remote sensing data classification.
• The network's ability to perceive and integrate spatial and spectral information enhances classification accuracy.
• The proposed method provides a robust solution for complex remote sensing classification tasks.