Double-branch feature fusion transformer for hyperspectral image classification

Area of Science:

• Remote Sensing
• Computer Vision
• Machine Learning

Background:

• Convolutional Neural Networks (CNNs) excel at local feature extraction in hyperspectral image (HSI) classification.
• CNNs struggle with long-distance dependencies and sequential properties inherent in HSI data.
• Existing CNN models do not fully leverage the rich spectral information in HSI.

Purpose of the Study:

• To propose a novel Double-Branch Feature Fusion Transformer model for enhanced HSI classification.
• To address the limitations of CNNs in capturing long-range spectral and spatial relationships in HSI.
• To improve the accuracy of HSI classification by effectively fusing spectral and spatial features.

Main Methods:

• Introduced Transformer architecture to leverage the sequential nature of HSI data.
• Developed a Double-Branch model to extract global spectral and spatial features independently.
• Implemented a feature fusion layer to combine spectral and spatial information.
• Designed attention modules to adaptively weigh spectral bands and pixels for classification.

Main Results:

• The proposed Double-Branch Feature Fusion Transformer model demonstrated superior performance compared to CNN-based models.
• Experiments on four public datasets confirmed the model's effectiveness in HSI classification accuracy.
• The model successfully captured long-distance spectral and spatial dependencies.

Conclusions:

• The Double-Branch Feature Fusion Transformer offers a significant advancement for hyperspectral image classification.
• Integrating Transformer architecture effectively addresses CNN limitations in handling HSI sequential properties.
• The model's ability to fuse spectral and spatial features leads to improved classification accuracy.