Related Experiment Video
Updated: May 29, 2026
13:19
Deep Neural Networks for Image-Based Dietary Assessment
Published on: March 13, 2021
Randomized neural network with adaptive forward regularization for online task-free class incremental learning
Junda Wang1, Minghui Hu2, Ning Li1
1School of Electronic Information and Electrical Engineering, Shanghai Jiao Tong University, Shanghai, 200240, China; Key Laboratory of System Control and Information Processing, Ministry of Education of China, Shanghai, 200240, China.
Summary
This study introduces edRVFL-kF-Bayes, a novel method for class incremental learning (CIL) that significantly reduces forgetting in online, task-free scenarios. It enhances knowledge retention and self-adapts to changing data distributions without replay or retraining.
Area of Science:
- Artificial Intelligence
- Machine Learning
- Deep Learning
Background:
- Class Incremental Learning (CIL) faces challenges with knowledge retention and catastrophic forgetting, especially in online, task-free (OTCIL) settings with non-i.i.d. data streams.
- Existing CIL methods struggle with continuous distribution drifting and heavy forgetting over long task sequences.
Purpose of the Study:
- To develop an efficient decision-making agent for the harsher online task-free CIL (OTCIL) scenario.
- To propose a method that avoids replay, retraining, and catastrophic forgetting while minimizing regret.
- To enhance robustness on non-i.i.d. streams and eliminate the need for manual tuning of hyperparameters.
Main Methods:
- Introduced ensemble deep random vector functional link network (edRVFL) with forward regularization (-F) to replace canonical Ridge (-R) for reduced regrets in OTCIL.
- Proposed edRVFL-kF to adjust forward knowledge intervention intensity for incremental updates, addressing distribution drifting.
- Developed plug-and-play edRVFL-kF-Bayes using online Bayesian learning for self-adaptive sub-learners, improving robustness and eliminating hyperparameter tuning.
Main Results:
- edRVFL with -F reduced regrets during OTCIL compared to canonical Ridge (-R).
- edRVFL-kF effectively avoided replay, retraining, and catastrophic forgetting, achieving lower regret than -R.
- edRVFL-kF-Bayes demonstrated superior robustness on non-i.i.d. streams and effective self-adaptation in OTCIL experiments on image datasets.
Conclusions:
- edRVFL-kF-Bayes offers an effective, robust, and adaptive solution for class incremental learning in challenging online, task-free scenarios.
- The proposed method significantly mitigates catastrophic forgetting and improves decision-making efficiency without requiring data replay or retraining.
- Experimental validation on image datasets confirms the efficacy and compatibility of edRVFL-kF-Bayes for long task streams with continuous distribution shifts.
Related Concept Videos
01:37Neural Regulation
Digestion begins with a cephalic phase that prepares the digestive system to receive food. When our brain processes visual or olfactory information about food, it triggers impulses in the cranial nerves innervating the salivary glands and stomach to prepare for food.
01:27Associative Learning
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
01:18Introduction to Learning
Learning is the process of acquiring knowledge or skills through practice or experience, leading to long-lasting behavioral changes. This acquisition occurs through interaction with the environment and requires practice or experience. For instance, mastering a skill such as surfing requires considerable practice and experience, highlighting the essential role of repeated interactions with the environment in learning.
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
01:14Avoidance Learning and Learned Helplessness
Avoidance learning and learned helplessness are critical concepts in understanding behavioral responses to negative stimuli.
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...