Generative binary memory: Pseudo-Replay class-Incremental learning on binarized embeddings.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Deep Neural Networks (DNNs) struggle to learn new classes without forgetting old ones, a problem known as Class-Incremental Learning (CIL).
• Existing CIL methods often require significant memory or struggle with multi-modal class distributions.

Purpose of the Study:

• Introduce Generative Binary Memory (GBM), a novel pseudo-replay approach for CIL.
• Enable DNNs to learn new classes while retaining existing knowledge effectively.

Main Methods:

• GBM utilizes Bernoulli Mixture Models (BMMs) to model multi-modal class distributions in a latent binary space.
• A feature binarizer allows GBM to be applied to any conventional DNN and natively supports Binary Neural Networks (BNNs).

Main Results:

• GBM achieved state-of-the-art results on CIFAR100 (+2.9%) and TinyImageNet (+1.5%) using ResNet-18.
• Outperformed emerging CIL methods for BNNs on CORE50, showing +3.1% accuracy and 4.7x memory reduction.

Conclusions:

• GBM offers an effective solution for CIL, improving accuracy and reducing memory footprint.
• The approach is versatile, applicable to standard DNNs and optimized for resource-constrained BNNs in embedded systems.