Dual prototypes for adaptive pre-trained model in class-incremental learning

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Class-incremental learning (CIL) aims to learn new classes sequentially while preserving knowledge of previously learned classes.
• Pre-trained model (PTM) based methods are effective but prone to catastrophic forgetting when fine-tuned on incremental tasks.
• Existing CIL approaches struggle to balance new knowledge acquisition with old knowledge retention.

Purpose of the Study:

• To propose a novel Dual-Prototype Network with Task-wise Adaptation (DPTA) for PTM-based CIL.
• To mitigate catastrophic forgetting in PTM-based CIL by introducing task-wise adaptation and dual prototypes.
• To improve the performance and knowledge retention capabilities of models in incremental learning scenarios.

Main Methods:

• Developed an adapter module for each incremental task to fine-tune the PTM.
• Introduced a center-adapt loss to promote centrally clustered and class-separable representations.
• Implemented a dual prototype network for test-time adapter selection and improved prediction using raw and augmented prototypes.

Main Results:

• DPTA consistently outperformed recent CIL methods by 1-5% across multiple benchmarks.
• Achieved approximately 3% improvement over state-of-the-art methods on the VTAB dataset.
• Demonstrated effective knowledge retention and improved class separability.

Conclusions:

• DPTA effectively addresses catastrophic forgetting in PTM-based CIL.
• The proposed dual-prototype network and task-wise adaptation significantly enhance incremental learning performance.
• DPTA offers a promising solution for robust and efficient class-incremental learning.