Feature Extraction for Classification of Hyperspectral and LiDAR Data Using Patch-to-Patch CNN.

Area of Science:

• Remote Sensing
• Geospatial Analysis
• Machine Learning

Background:

• Multisensor fusion is crucial for Earth observation, leveraging hyperspectral images (HSIs) for spectral data and Light Detection and Ranging (LiDAR) for elevation data.
• Combining HSI and LiDAR data offers synergistic advantages for improved classification accuracy in remote sensing applications.

Purpose of the Study:

• To propose an unsupervised feature extraction framework, the Patch-to-Patch Convolutional Neural Network (PToP CNN), for the collaborative classification of HSI and LiDAR data.
• To develop a novel three-tower PToP mapping for merging multiscale features between HSI and LiDAR data sources.

Main Methods:

• A three-tower PToP mapping was designed to learn accurate representations from HSI to LiDAR data.
• Features from different hidden layers of the PToP CNN were integrated to achieve deep fusion.
• Extracted features were concatenated and fed into fully connected layers for classification.

Main Results:

• The proposed PToP CNN framework demonstrated superior classification performance on benchmark remote sensing datasets.
• Experimental results showed significant improvements compared to state-of-the-art classifiers like two-branch CNN and context CNN.

Conclusions:

• The PToP CNN framework effectively achieves unsupervised feature extraction and deep fusion of HSI and LiDAR data.
• The proposed method offers a powerful tool for enhancing classification accuracy in Earth observation applications.