Efficient Perturbation Inference and Expandable Network for continual learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Neural networks often forget previous knowledge when learning new tasks.
• This phenomenon, known as catastrophic forgetting, limits their real-world applicability.

Purpose of the Study:

• To develop a novel neural network architecture that overcomes catastrophic forgetting in class incremental learning.
• To improve model robustness and performance without significantly increasing parameter count.

Main Methods:

• Proposing the Efficient Perturbation Inference and Expandable Network (EPIE-Net).
• Dynamically expanding lightweight task-specific decoders for new classes.
• Utilizing a mixed-label uncertainty strategy and averaging perturbed sample probabilities at inference.

Main Results:

• EPIE-Net consistently outperforms existing methods in class incremental learning benchmarks.
• Achieved 76.33% average accuracy and 65.93% last accuracy on CIFAR-100 (10 steps).
• Demonstrated superior performance with a significantly lower parameter count (3.46M average parameters).

Conclusions:

• EPIE-Net effectively mitigates catastrophic forgetting in neural networks.
• The proposed method offers a parameter-efficient solution for incremental learning.
• EPIE-Net shows strong potential for real-world applications requiring continuous learning.