SpectralGPT: Spectral Remote Sensing Foundation Model.

Area of Science:

• Geoscience
• Artificial Intelligence
• Remote Sensing

Background:

• Foundation models excel at self-supervised visual representation learning.
• Existing models primarily process RGB images, neglecting valuable spectral data in remote sensing.
• Spectral data offers crucial information for understanding remote sensing scenes.

Purpose of the Study:

• To introduce the first universal foundation model specifically designed for spectral remote sensing images.
• To address the gap in self-supervised learning for spectral remote sensing data.
• To leverage advanced transformer architectures for enhanced spatial-spectral feature extraction.

Main Methods:

• Developed SpectralGPT, a novel 3D generative pretrained transformer (GPT) model.
• Employed progressive training to handle diverse remote sensing data (varying sizes, resolutions, time series).
• Utilized 3D token generation for spatial-spectral coupling and multi-target reconstruction for spectrally sequential patterns.

Main Results:

• Trained SpectralGPT on one million spectral remote sensing images, resulting in models with over 600 million parameters.
• Demonstrated significant performance improvements across four downstream tasks: scene classification, semantic segmentation, and change detection.
• Showcased the model's ability to effectively utilize extensive remote sensing Big Data.

Conclusions:

• SpectralGPT represents a significant advancement in self-supervised learning for spectral remote sensing.
• The model holds substantial potential for improving geoscience applications reliant on remote sensing Big Data.
• This work paves the way for more sophisticated analysis of spectral remote sensing imagery.