Pursuing Better Representations: Balancing Discriminability and Transferability for Few-Shot Class-Incremental...

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Few-Shot Class-Incremental Learning (FSCIL) addresses the challenge of learning new classes with limited data while preserving knowledge of existing classes.
• Existing FSCIL methods often freeze pre-trained backbones, focusing on representation learning, particularly Self-Supervised Contrastive Learning (SSCL).
• A key limitation is the trade-off between representation transferability and discriminability, hindering simultaneous high performance on base and novel classes.

Purpose of the Study:

• To propose BR-FSCIL, a novel representation learning framework for FSCIL that overcomes the transferability-discriminability trade-off.
• To enhance model generalization to novel classes while mitigating catastrophic forgetting of previously learned information.
• To achieve a balanced representation learning that is both transferable and discriminative.

Main Methods:

• Introduced Hierarchical Contrastive Learning (HierCon) to leverage label information for modeling hierarchical feature relationships, enhancing discriminability alongside transferability.
• Proposed an Alignment Modulation (AM) loss to facilitate inter-class knowledge sharing, improving adaptability to novel classes.
• Optimized representations at both intra-class and inter-class levels.

Main Results:

• BR-FSCIL achieved final session accuracies of 53.83% on mini-ImageNet, 53.04% on CIFAR100, and 62.60% on CUB200.
• The proposed HierCon and AM loss effectively balanced representation discriminability and transferability.
• Demonstrated superior performance compared to existing methods in the FSCIL scenario.

Conclusions:

• BR-FSCIL effectively addresses the limitations of current FSCIL representation learning methods.
• The framework successfully balances discriminability and transferability, leading to improved performance on both base and novel classes.
• The proposed HierCon and AM loss are effective components for advancing FSCIL research.