NTK-Guided Few-Shot Class Incremental Learning.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Computer Vision

Background:

• Few-Shot Class Incremental Learning (FSCIL) methods face challenges in retaining knowledge from previous tasks (anti-amnesia).
• Existing FSCIL approaches often struggle with catastrophic forgetting, limiting their real-world applicability.
• Robust anti-amnesia is crucial for developing effective lifelong learning systems.

Purpose of the Study:

• To introduce a novel conceptualization of anti-amnesia in FSCIL based on mathematical generalization using the Neural Tangent Kernel (NTK).
• To develop a method that ensures optimal NTK convergence and minimizes NTK-related generalization loss for improved cross-task generalization.
• To enhance the theoretical generalization capabilities of FSCIL models.

Main Methods:

• Leveraging the Neural Tangent Kernel (NTK) perspective to analyze and improve anti-amnesia.
• Implementing a meta-learning mechanism for global NTK convergence within an expanded network architecture.
• Reducing NTK-related generalization loss through self-supervised pre-training, curricular alignment, and dual NTK regularization for convolutional and linear layers.

Main Results:

• The proposed NTK-FSCIL method demonstrates robust NTK properties, ensuring optimal convergence and stability.
• Significant reduction in NTK-related generalization loss, leading to enhanced theoretical generalization.
• Achieved state-of-the-art performance on popular FSCIL benchmark datasets, with accuracy improvements ranging from 2.9% to 9.3%.

Conclusions:

• The NTK-based approach provides a strong theoretical foundation for anti-amnesia in FSCIL.
• The proposed methods effectively mitigate forgetting and improve knowledge transfer across tasks.
• NTK-FSCIL offers a promising direction for developing more stable and accurate lifelong learning systems.