Enhanced Cloud Detection Using a Unified Multimodal Data Fusion Approach in Remote Images

Area of Science:

• Computer Science
• Remote Sensing
• Artificial Intelligence

Background:

• Multimodal cloud detection faces challenges in complex network architectures and computational inefficiency with varying data modalities.
• Existing models often require significant architectural adjustments for new data types, increasing development costs.

Purpose of the Study:

• To propose M2Cloud, an efficient and unified model for multimodal cloud detection capable of processing an arbitrary number of modalities.
• To develop a novel multimodal data fusion method that reduces computational costs and enhances efficiency.
• To create a flexible and generalizable fusion module for seamless integration into various network architectures.

Main Methods:

• M2Cloud employs a novel multimodal data fusion approach that avoids modality-specific architectural changes.
• Feature extraction utilizes shared, independent weights per modality to preserve inherent characteristics.
• Cosine similarity is used for adaptive learning of complementary features, minimizing redundant information.

Main Results:

• M2Cloud demonstrates state-of-the-art (SOTA) performance on the WHUS2-CD and WHUS2-CD+ multimodal datasets.
• The model achieves high effectiveness in unified multimodal cloud detection.
• The proposed fusion module exhibits strong generalization and plug-and-play capabilities.

Conclusions:

• M2Cloud offers an efficient and unified solution for multimodal cloud detection, adaptable to any number of data modalities.
• The novel fusion method significantly reduces incremental computing costs and enhances overall system efficiency.
• This research provides valuable technical support and new insights for multimodal data fusion and cloud detection applications.