An Adaptive Migration Collaborative Network for Multimodal Image Classification.

Area of Science:

• Remote Sensing
• Computer Vision
• Machine Learning

Background:

• Multispectral (MS) and panchromatic (PAN) images possess distinct properties, leading to significant representational gaps and feature space disparities.
• Independent feature extraction hinders collaborative classification, and varying layer capabilities complicate the modeling of objects at different scales.

Purpose of the Study:

• To propose an adaptive migration collaborative network (AMC-Net) for classifying multimodal remote-sensing images.
• To dynamically transfer dominant attributes, reduce inter-modal gaps, identify optimal shared layer representations, and fuse features with varying representational capacities.

Main Methods:

• Input preprocessing combines Principal Component Analysis (PCA) and Nonsubsampled Contourlet Transformation (NSCT) for attribute migration and image similarity enhancement.
• A Feature Progressive Migration Fusion Unit (FPMF-Unit) with Correlation Coefficient Analysis (CCA) enables adaptive learning of shared features.
• An Adaptive Layer Fusion Mechanism Module (ALFM-Module) fuses features from different layers to model multi-scale object dependencies.
• The loss function incorporates correlation coefficient calculation to promote convergence to the global optimum.

Main Results:

• The proposed AMC-Net effectively reduces the representational gap between MS and PAN images.
• The FPMF-Unit and ALFM-Module facilitate adaptive feature sharing and multi-layer fusion.
• The network demonstrates competitive performance in multimodal remote-sensing image classification.

Conclusions:

• AMC-Net offers a robust solution for multimodal remote-sensing image classification by addressing feature representation gaps and fusion challenges.
• The adaptive migration and fusion strategies contribute to improved classification accuracy.
• The developed framework provides a valuable tool for remote-sensing data analysis.