Cycle Translation-Based Collaborative Training for Hyperspectral-RGB Multimodal Change Detection.

Area of Science:

• Remote Sensing
• Computer Vision
• Geospatial Analysis

Background:

• Hyperspectral image change detection (HSI-CD) relies on homogenous multi-temporal HSIs, which are difficult to acquire.
• HSI-RGB multimodal CD addresses HSI availability issues by integrating RGB data, but faces challenges due to differing imaging mechanisms.

Purpose of the Study:

• To develop a robust HSI-RGB multimodal change detection method overcoming modality differences.
• To enable collaborative learning of complementary change information from HSI and RGB data.

Main Methods:

• A cycle translation-based collaborative training (co-training) framework is proposed for HSI-RGB multimodal CD.
• A cross-modal guided CycleGAN-based image translation module mitigates modality differences via bi-directional translation.
• A spatial-spectral interactive co-training CD module facilitates iterative cross-modal information exchange for difference feature extraction.

Main Results:

• The proposed method demonstrates superior performance compared to existing CD techniques on both real and synthetic datasets.
• The approach effectively extracts complementary change information by leveraging cross-modal interactions.
• Successful mitigation of modality differences enables more accurate change detection.

Conclusions:

• The developed cycle translation and co-training method offers a significant advancement in HSI-RGB multimodal change detection.
• The approach provides a robust solution for identifying land cover changes using diverse data sources.
• The study contributes a new public HSI-RGB multimodal dataset and open-source code.