Multimodal Contrastive Learning for Remote Sensing Image Feature Extraction Based on Relaxed Positive Samples.

Area of Science:

• Remote Sensing
• Computer Vision
• Machine Learning

Background:

• Traditional multimodal contrastive learning enforces strict identity constraints between text and images.
• Remote sensing image complexity and rich ancillary information challenge these constraints.
• Existing methods may be insufficient for effectively describing and analyzing remote sensing data.

Purpose of the Study:

• To propose a novel multimodal contrastive learning method for remote sensing image feature extraction.
• To address the limitations of strict identity constraints in current approaches.
• To leverage the unique characteristics of remote sensing images for improved feature representation.

Main Methods:

• Introduced a "positive sample tripartite relaxation" in multimodal contrastive learning.
• Relaxed input constraints by using learnable parameters in language and image branches for flexible descriptions and ancillary information extraction.
• Enabled multimodal alignment of various features, specifically aligning semantic information with corresponding image regions for relaxed local feature extraction under semantic constraints.

Main Results:

• Achieved 91.1% accuracy on the PatternNet dataset with one-shot learning.
• Validated the proposed method on four diverse remote sensing datasets.
• Demonstrated improved feature extraction capabilities for remote sensing images.

Conclusions:

• The proposed relaxed multimodal contrastive learning method effectively extracts features from remote sensing images.
• The approach successfully overcomes limitations of strict identity constraints by introducing flexibility in inputs and feature alignment.
• This method offers a promising direction for advanced remote sensing image analysis and understanding.