Uncertainty-Aware Distillation for Semi-Supervised Few-Shot Class-Incremental Learning.

Area of Science:

• Computer Vision
• Machine Learning
• Artificial Intelligence

Background:

• Few-shot class-incremental learning (FSCIL) aims to learn new classes with limited data without forgetting previous knowledge.
• Semi-supervised learning (SSL) leverages unlabeled data to improve model performance, but its application in FSCIL (Semi-FSCIL) is underexplored.
• A key challenge is adapting SSL techniques to the FSCIL setting, specifically addressing the issue of model adaptability.

Purpose of the Study:

• To address the adaptability challenge of semi-supervised learning in FSCIL tasks.
• To present a novel and efficient Semi-FSCIL framework named uncertainty-aware distillation with class-equilibrium (UaD-ClE).
• To improve the performance of incremental learning by effectively utilizing unlabeled data.

Main Methods:

• The proposed UaD-ClE framework integrates two modules: uncertainty-aware distillation (UaD) and class equilibrium (ClE).
• The ClE module utilizes class-balanced self-training (CB_ST) to prevent easy classes from dominating pseudo-label generation when incorporating unlabeled data.
• The UaD module employs uncertainty-guided knowledge refinement and adaptive distillation to transfer knowledge from the reference model.

Main Results:

• Comprehensive experiments were conducted on three benchmark datasets.
• The UaD-ClE method demonstrated significant improvements in boosting the adaptability of unlabeled data within FSCIL tasks.
• The proposed framework effectively mitigates catastrophic forgetting and overfitting in incremental learning scenarios.

Conclusions:

• The UaD-ClE framework provides an effective solution for Semi-FSCIL by improving model adaptability.
• The integration of class-balanced self-training and uncertainty-aware distillation enhances the utilization of unlabeled data.
• The method shows promise for advancing few-shot class-incremental learning research and applications.