Miles: Metric Learning With Expandable Subspace for Pre-Trained Model-Based Class-Incremental Learning

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Class Incremental Learning (CIL) aims to learn new data sequentially without forgetting past knowledge.
• Pre-trained models (PTMs) offer efficient task adaptation but existing CIL methods struggle with performance-cost trade-offs.
• Current PTM-based CIL methods either cause catastrophic forgetting or increase computational load via task-specific branches.

Purpose of the Study:

• To propose an efficient PTM-based Class Incremental Learning method that balances performance and computational expenditure.
• To introduce a novel approach for parameter space expansion guided by prior knowledge.
• To address the limitations of existing CIL methods in adapting PTMs.

Main Methods:

• MetrIc Learning with Expandable Subspace (Miles) leverages prior information from PTMs for guided parameter space expansion.
• Decouples learnable modules from the PTM backbone, utilizing intermediate features for flexible parameter growth.
• Employs a central loss for clustering new categories towards prototypes and a regularization term for metric equilibrium.

Main Results:

• Miles achieves state-of-the-art performance across various Class Incremental Learning scenarios.
• Demonstrates superior results in balancing learning new concepts with preserving old knowledge.
• Validates effectiveness on six benchmark datasets, showcasing significant improvements over existing methods.

Conclusions:

• Miles offers an effective solution for Class Incremental Learning using pre-trained models.
• The proposed method successfully navigates the trade-off between performance and computational efficiency.
• Miles provides a flexible and powerful framework for continual learning applications.