Multi-Level Collaborative Distillation Meets Global Workspace Model: A Unified Framework for OCIL.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Science

Background:

• Online Class-Incremental Learning (OCIL) faces challenges in balancing stability and plasticity with non-i.i.d. data streams.
• Existing replay-based methods are limited by memory constraints, and ensemble methods often lack stability.

Purpose of the Study:

• To propose a novel approach inspired by Global Workspace Theory (GWT) to enhance OCIL.
• To improve the balance between stability and plasticity in OCIL models under memory constraints.

Main Methods:

• Introduced a Global Workspace Model (GWM) that fuses student model parameters to capture historical learning.
• Implemented a GWM broadcasting mechanism for stabilizing learning and promoting cross-task consistency.
• Utilized multi-level collaborative distillation for peer-to-peer consistency and knowledge preservation.

Main Results:

• The proposed method significantly improves the performance of OCIL models across various memory budgets.
• Demonstrated enhanced stability and plasticity in student models.
• Achieved substantial performance gains on three standard OCIL benchmarks.

Conclusions:

• The GWM approach effectively addresses the stability-plasticity dilemma in OCIL.
• The method offers a robust solution for continuous learning in real-world scenarios with limited memory.
• The approach provides a promising direction for advancing OCIL research.