Adaptive graph contrastive learning with hard negative mining for multimodal hyperspectral and LiDAR classification.

Area of Science:

• Remote Sensing
• Computer Vision
• Machine Learning

Background:

• Joint classification of hyperspectral imagery (HSI) and light detection and ranging (LiDAR) data is crucial but challenging due to data heterogeneity.
• Existing graph neural network (GNN) methods often require extensive labeled data, limiting their use in remote sensing where labels are scarce.

Purpose of the Study:

• To propose a self-supervised graph framework, adaptive graph contrastive learning (AGCL), for robust HSI and LiDAR classification.
• To address the challenges of multimodal fusion and feature modeling in label-scarce scenarios.

Main Methods:

• Developed AGCL, a self-supervised graph framework utilizing input-conditioned neighborhood selection for adaptive graph construction.
• Implemented dynamic affinity matrices for flexible message passing and a hard negative mining strategy for effective contrastive learning.
• Employed joint optimization of intra-modal consistency, cross-modal alignment, and graph topology reconstruction during self-supervised pretraining.

Main Results:

• The proposed AGCL framework demonstrated effectiveness in HSI and LiDAR classification across three benchmark datasets.
• Self-supervised pretraining enabled robust feature learning from heterogeneous multimodal data without labeled samples.
• Learned representations were successfully transferred to downstream classification tasks via supervised fine-tuning.

Conclusions:

• AGCL offers a powerful self-supervised approach for joint HSI and LiDAR classification, overcoming limitations of supervised methods in label-scarce environments.
• The adaptive graph construction and contrastive learning strategies enhance multimodal fusion and feature representation for remote sensing applications.