Enhancing Feature Learning With Hard Samples in Mutual Learning for Online Class Incremental Learning.

Area of Science:

• Machine Learning
• Computer Science
• Artificial Intelligence

Background:

• Online Class-Incremental Learning (OCIL) addresses learning from non-i.i.d. data streams in a single pass, crucial for real-time applications.
• The one-epoch training constraint in OCIL often leads to under-fitting and instability due to learning non-essential features.

Purpose of the Study:

• To investigate the use of hard samples for improving data variability and feature learning in OCIL.
• To develop an OCIL formulation that enhances generalization ability within single-epoch training constraints.

Main Methods:

• Introduced a scoring function to identify and generate high-value samples, improving data variability.
• Employed strong data augmentation to increase the proportion of high-score samples.
• Designed a mutual learning framework with two identical networks and a collaborative learning mechanism for feature and probability alignment.

Main Results:

• The proposed method effectively generates high-value samples and optimizes the OCIL model.
• Data augmentation proved effective in generating a higher proportion of high-score samples.
• The collaborative learning mechanism promoted interaction between the two networks, enhancing performance.

Conclusions:

• The developed OCIL method significantly improves model stability and generalization by effectively utilizing hard samples.
• The approach achieves superior performance compared to state-of-the-art methods on widely used OCIL datasets.
• The study demonstrates the efficacy of combining hard sample generation, data augmentation, and mutual learning for OCIL.