Prototype-Based Meta-Prompt Tuning: Toward Rehearsal-Free Few-Shot Class-Incremental Learning for Multimodal Remote...

Area of Science:

• Remote Sensing
• Computer Vision
• Machine Learning

Background:

• Multi-modal remote sensing data classification performs well within fixed label sets.
• Dynamic surface conditions cause land cover class variations over time.
• Retraining models for new classes incurs high computational costs and data privacy issues.

Purpose of the Study:

• To propose a novel framework, prototype-based meta-prompt tuning (PMPT), for incremental learning in multi-modal remote sensing.
• To address the challenges of dynamic land cover changes, computational cost, and data privacy in existing classification models.
• To enable models to adapt to new classes with limited data while preserving historical knowledge.

Main Methods:

• Developed the PMPT framework featuring a meta-learning backbone and an incrementally updated nearest-class-mean (NCM) classifier.
• Froze the backbone after initial training on base classes, fine-tuning only session-relevant visual prompts for incremental adaptation.
• Introduced an incremental prototype contrastive loss to mitigate semantic drift and prototype overlap.

Main Results:

• The PMPT framework effectively fine-tunes visual prompts for incremental class adaptation, preserving historical knowledge via prototype embeddings.
• The NCM classifier, combined with frozen backbone and prompt tuning, alleviates knowledge forgetting and overfitting.
• Demonstrated effectiveness on multimodal remote sensing datasets, showing successful classification of unknown samples with limited incremental data.

Conclusions:

• PMPT offers an effective solution to the stability-plasticity dilemma in incremental learning for remote sensing.
• The framework successfully classifies new land cover classes with minimal data and computational overhead.
• PMPT enhances the adaptability and efficiency of multi-modal remote sensing data classification systems.