Low-data cross-modal adaptation for remote sensing with proxy-enhanced multi-granularity feature caching.

Area of Science:

• Computer Vision
• Remote Sensing
• Artificial Intelligence

Background:

• Vision-language models (VLMs) offer potential for open-vocabulary recognition in remote sensing.
• Current VLMs struggle with generic prompts, limited datasets, and domain-specific features in remote sensing.
• Existing cross-modal adaptation methods face challenges in low-data scenarios.

Purpose of the Study:

• To propose a proxy-enhanced multi-granularity feature caching adaptation framework for cross-modal remote sensing imagery.
• To address limitations of generic prompts, data scarcity, and feature discriminability in remote sensing VLMs.
• To improve performance in low-data and few-shot remote sensing scenarios.

Main Methods:

• Developed an LLM-augmented prompt module to transform class labels into descriptive attributes.
• Implemented a proxy-enhanced semantic calibration mechanism for class-level visual proxies and pseudo-label generation.
• Utilized a multi-granularity feature cache storing patch-level texture and scene-level topological features.
• Integrated cached features with zero-shot CLIP predictions during inference.

Main Results:

• The proposed framework demonstrates stable performance in few-shot scenarios where traditional fine-tuning fails.
• Outperformed existing cross-modal adaptation approaches on multiple benchmark remote sensing datasets.
• Enhanced semantic grounding through LLM prompts and proxy-based support sets.
• Mitigated domain-specific representation gaps via feature caching and proxy calibration.

Conclusions:

• The framework effectively improves cross-modal adaptation for remote sensing in low-data settings.
• The integration of LLM prompts, proxy calibration, and feature caching addresses key VLMs limitations.
• The method shows significant promise for advancing open-vocabulary recognition in remote sensing applications.