Multimodal Quaternion Representation Network for Multisource Remote Sensing Data Classification.

Area of Science:

• Remote Sensing
• Computer Vision
• Data Fusion

Background:

• Effective integration of hyperspectral images (HSIs) and light detection and ranging (LiDAR) data is crucial for Earth observation.
• Challenges include insufficient information utilization and feature heterogeneity in multisource remote sensing (RS) data.

Purpose of the Study:

• To propose a novel multimodal quaternion representation network (MMQRN) for enhanced multisource RS data classification.
• To address limitations in feature fusion and utilization for improved Earth observation.

Main Methods:

• Developed a multimodal quaternion representation (MMQR) to model complex nonlinear interactions among complementary features.
• Designed a multimodal feature cross-fusion (MFCF) framework for integrating multisource, multimodal, and multilevel features.
• Utilized a quaternion convolutional transformer network (QCTN) to capture global and local spatial-spectral information.

Main Results:

• The proposed MMQRN demonstrated superior performance compared to existing state-of-the-art classification methods.
• Experiments on three multisource RS datasets validated the effectiveness of the MMQRN approach.

Conclusions:

• The MMQRN effectively fuses HSIs and LiDAR data, overcoming challenges in feature heterogeneity and information utilization.
• This network offers a significant advancement in multisource remote sensing data classification for Earth observation.