Dual selective fusion transformer network for hyperspectral image classification.

Area of Science:

• Remote Sensing
• Computer Vision
• Machine Learning

Background:

• Transformer models show promise in hyperspectral image (HSI) classification.
• Existing models struggle with HSI's diverse land cover and spectral information due to fixed receptive fields and suboptimal self-attention fusion.

Purpose of the Study:

• To propose a novel Dual Selective Fusion Transformer Network (DSFormer) for improved HSI classification.
• To address limitations of fixed receptive fields and invalid self-attention features in current Transformer models for HSI analysis.

Main Methods:

• Developed a Dual Selective Fusion Transformer Network (DSFormer) for joint spatial and spectral contextual modeling.
• Introduced Kernel Selective Fusion Transformer Block (KSFTB) for adaptive multi-scale feature fusion.
• Implemented Token Selective Fusion Transformer Block (TSFTB) for strategic token selection in self-attention fusion.

Main Results:

• DSFormer significantly improved land cover classification accuracy on four benchmark HSI datasets.
• Achieved overall accuracies of 96.59% (Pavia University), 97.66% (Houston), 95.17% (Indian Pines), and 94.59% (Whu-HongHu).
• Demonstrated performance improvements of 3.19%, 1.14%, 0.91%, and 2.80% over previous state-of-the-art methods.

Conclusions:

• DSFormer effectively models spatial and spectral contexts by selectively fusing features across scales.
• The proposed network enhances the identification of diverse HSI objects, outperforming existing methods.
• The DSFormer architecture offers a promising advancement for accurate hyperspectral image classification.