RingMoE: Mixture-of-Modality-Experts Multi-Modal Foundation Models for Universal Remote Sensing Image Interpretation.

Area of Science:

• Earth and Planetary Sciences
• Computer Science
• Artificial Intelligence

Background:

• Foundation models have advanced self-supervised visual representation learning.
• Remote sensing (RS) applications are limited by models handling single or few modalities.
• Multi-modal RS data (optical, SAR, multi-spectral) offer complementary insights, reducing ambiguity.

Purpose of the Study:

• To introduce RingMoE, a unified multi-modal RS foundation model addressing the limitations of single-modality approaches.
• To leverage the complementary nature of diverse RS data for improved analysis.
• To facilitate efficient deployment of advanced RS models in Earth observation.

Main Methods:

• Developed RingMoE, a 14.7 billion parameter multi-modal RS foundation model pre-trained on 400 million images.
• Implemented a hierarchical Mixture-of-Experts (MoE) architecture for specialized and collaborative expert modeling.
• Incorporated physics-informed self-supervised learning and dynamic expert pruning for adaptive compression.

Main Results:

• RingMoE achieved state-of-the-art (SOTA) performance across 23 benchmarks in six key RS tasks.
• Demonstrated superior adaptability from single-modal to multi-modal scenarios.
• Successfully deployed and trialed in sectors including emergency response and land management.

Conclusions:

• RingMoE effectively bridges the gap in multi-modal RS foundation models.
• The model's architecture and training enable robust performance and efficient deployment.
• RingMoE shows significant potential for advancing various Earth observation applications.