Deep Hierarchical Vision Transformer for Hyperspectral and LiDAR Data Classification.

Area of Science:

• Remote Sensing
• Computer Vision
• Machine Learning

Background:

• Current remote sensing data classification methods struggle with integrating heterogeneous features from multiple data sources like hyperspectral and LiDAR.
• Limitations in feature representation and information fusion hinder accurate collaborative classification of multi-modality remote sensing data.

Purpose of the Study:

• To develop a novel Deep Hierarchical Vision Transformer (DHViT) architecture for joint classification of hyperspectral and LiDAR data.
• To overcome limitations in heterogeneous feature representation and information fusion for improved remote sensing data classification.

Main Methods:

• The DHViT architecture leverages the transformer network's self-attention mechanism for modeling long-range dependencies and generalization.
• A spectral sequence transformer handles spectral dimension dependencies in hyperspectral images.
• A spatial hierarchical transformer extracts hierarchical spatial features from hyperspectral and LiDAR data.
• Cross-attention (CA) feature fusion adaptively integrates heterogeneous multi-modality features.

Main Results:

• The DHViT model achieved high average overall classification accuracies: 99.58%, 99.55%, and 96.40% on three benchmark datasets.
• Comparative experiments and ablation studies validated the model's effectiveness and superior performance.
• The CA feature fusion pattern demonstrated adaptive and dynamic fusion of heterogeneous features, enhancing classification.

Conclusions:

• The proposed DHViT architecture significantly advances joint classification of hyperspectral and LiDAR remote sensing data.
• The method effectively addresses challenges in multi-modality data fusion, leading to state-of-the-art classification performance.
• DHViT offers a powerful and effective solution for complex remote sensing classification tasks.