Attribute-guided feature fusion network with knowledge-inspired attention mechanism for multi-source remote sensing...

Area of Science:

• Remote Sensing
• Geospatial Analysis
• Artificial Intelligence

Background:

• Accurate land use and land cover (LULC) classification is crucial for environmental monitoring and management.
• Single-modal data often lacks sufficient information for precise classification, especially in complex environments.
• Multi-modal data, like hyperspectral images (HSIs) and light detection and ranging (LiDAR), offer complementary information to improve LULC classification.

Purpose of the Study:

• To propose an attribute-guided feature fusion network with knowledge-inspired attention mechanisms (AFNKA) for improved multi-modal LULC classification.
• To address limitations in existing methods by incorporating data-specific knowledge, such as spectral mixtures in HSIs and spatial scales in LiDAR data.
• To enhance feature representation and fusion by considering the distinct physical attributes of HSI (spectral) and LiDAR (elevation) data.

Main Methods:

• Development of a knowledge-inspired attention mechanism to extract enhanced features from HSIs and LiDAR data.
• Introduction of a novel adaptive cosine estimator (ACE) based attention module to learn discriminative features, leveraging spatial-spectral correlations in HSIs.
• Design of two attribute-guided fusion modules to selectively aggregate multi-modal features, exploiting the correlations between HSI's spatial-spectral properties and LiDAR's spatial-elevation properties.

Main Results:

• The proposed AFNKA network significantly outperforms existing state-of-the-art methods in LULC classification tasks.
• Quantitative results on multiple multi-source datasets demonstrate the effectiveness of the attribute-guided fusion and knowledge-inspired attention mechanisms.
• The method successfully utilizes the complementary nature of HSI and LiDAR data for more accurate classification.

Conclusions:

• AFNKA provides a superior approach for multi-modal LULC classification by intelligently fusing HSI and LiDAR data.
• The knowledge-inspired attention and attribute-guided fusion strategies effectively address the challenges of spectral mixtures and varying spatial scales.
• This research advances the field of remote sensing by offering a more robust and accurate method for understanding land surface characteristics.