Batch self-organizing memory neural network for continual supervised learning

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Continual learning in artificial neural networks (ANNs) aims to mimic human intelligence by learning new tasks without forgetting previous ones.
• Dynamic architecture strategies expand network capacity for new tasks but often struggle with appropriate strategy selection.
• Task identifiers are typically required to guide parameter or component selection for each task, posing a limitation.

Purpose of the Study:

• Propose a Batch Self-organizing Memory Neural Network (Batch SOMNN) for continual supervised learning.
• Address the challenge of selecting appropriate network expansion strategies and the need for task identifiers.
• Enhance the efficiency and cost-effectiveness of network expansion in deep learning models.

Main Methods:

• Introduce a Batch Supervised Competitive Learning (BSCL) algorithm for learning new tasks without identifiers by detecting distributional shifts.
• Dynamically generate new regions to accommodate new data while preserving existing memory.
• Employ a Memory Correction (MC) module using forgetting curves and adaptive thresholds to retain valuable information.

Main Results:

• Batch SOMNN demonstrates superior performance compared to strong baselines in continual learning scenarios.
• The proposed BSCL algorithm effectively learns new tasks without requiring task identifiers.
• The MC module enhances the efficiency and cost-effectiveness of network expansion.

Conclusions:

• Batch SOMNN offers a robust solution for continual supervised learning, overcoming limitations of existing methods.
• The extension to Deep Batch SOMNN (DB-SOMNN) with deep feature extraction achieves state-of-the-art results on complex datasets.
• This work advances the field of continual learning by improving adaptability and performance in ANNs.