Leveraging joint incremental learning objective with data ensemble for class incremental learning.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Class-incremental learning models face catastrophic forgetting, losing knowledge of older classes when trained on new data.
• Image orientation significantly impacts model prediction accuracy, revealing varied forgetting rates for different orientations of the same image.

Purpose of the Study:

• To propose a novel data-ensemble approach combined with a joint-incremental learning objective (JILO) to mitigate catastrophic forgetting in class-incremental learning.
• To demonstrate the effectiveness of the proposed approach in improving the retention of previously learned class information.

Main Methods:

• A data-ensemble method combining predictions from different image orientations to preserve past knowledge.
• A novel training approach using JILO, integrating class-incremental and data-incremental learning objectives.
• Empirical evaluation on state-of-the-art class-incremental learning methods, including AANets on the CIFAR-100 dataset.

Main Results:

• The proposed data-ensemble approach, enabled by JILO, significantly reduces the rate of forgetting in deep learning models.
• Significant performance improvements were observed on the CIFAR-100 dataset, with absolute margins up to 4.28% for the AANets method.
• The efficacy of JILO as vital to the data-ensemble approach was empirically demonstrated.

Conclusions:

• The proposed approach effectively combats catastrophic forgetting in class-incremental learning by leveraging image orientation variations and a novel joint-incremental learning objective.
• The method offers a significant advancement in maintaining model performance across sequential learning phases.
• This work establishes a new direction for improving the robustness of incremental learning systems.