Model Attention Expansion for Few-Shot Class-Incremental Learning | JoVE Visualize

Area of Science:

Artificial Intelligence
Machine Learning
Computer Vision

Background:

Few-Shot Class-Incremental Learning (FSCIL) enables models to learn new classes with limited data without forgetting prior knowledge.
Existing FSCIL methods suffer from 'supervision collapse,' where base class features dominate, hindering novel class representation and overall model capability.
This limitation restricts the cognitive abilities of AI models in dynamic learning environments.

Purpose of the Study:

To propose a novel framework, Model aTtention Expansion for Few-Shot Class-Incremental Learning (MTE-FSCIL), to address supervision collapse in FSCIL.
To enhance model attention fields for improved knowledge transferability while preserving discriminative capabilities for base classes.
To develop strategies for continuous knowledge incorporation and cross-task learning.

Main Methods:

A dual-stage training strategy involving pre-training and meta-training.
Introduction of the Reserver (RS) loss during pre-training to amplify feature map activations, expanding global perception and reducing reliance on class-specific features.
Implementation of the Repeller (RP) loss during meta-training to promote representational diversity and improve sample uniqueness recognition by scattering intra-class samples.
A Transformational Adaptation (TA) strategy for seamless integration of new knowledge from downstream tasks.

Main Results:

The MTE-FSCIL framework demonstrated superior performance compared to state-of-the-art methods across benchmark datasets including mini-ImageNet, CIFAR100, and CUB200.
The proposed RS loss effectively expands global perception and mitigates over-reliance on class-specific features.
The RP loss successfully enhances representational variation and improves the recognition of sample uniqueness.
The TA strategy facilitates effective cross-task knowledge transfer.

Conclusions:

MTE-FSCIL effectively overcomes the supervision collapse challenge in Few-Shot Class-Incremental Learning.
The proposed framework significantly improves model performance and cognitive capabilities in incremental learning scenarios.
The MTE-FSCIL framework offers a promising direction for developing more robust and adaptable AI systems.