Randomized neural network with adaptive forward regularization for online task-free class incremental learning

Class incremental learning (CIL) requires an agent to learn distinct tasks consecutively with knowledge retention against forgetting. Problems impeding the practice of CIL methods are twofold: (1) prompt update on non-i.i.d batch streams without boundary, namely the harsher online task-free CIL (OTCIL) scenario; (2) CIL methods suffer from heavy forgetting on learning long task streams, as shown in Fig. 1(a). To achieve efficient decision-making, the ensemble deep random vector functional link network (edRVFL) with forward regularization (-F) is proposed to replace the canonical Ridge (-R), reducing more regrets during OTCIL. Considering continuous distribution drifting on long stream, we further propose edRVFL-kF to adjust the intervention intensity of forward knowledge and derive incremental updates. edRVFL-kF can effectively avoid replay, retraining, and catastrophic forgetting while achieving lower regret over -R. Moreover, to improve robustness on non-i.i.d stream and eliminate intractable tuning of -kF, we rebuild with online Bayesian learning and propose the plug-and-play edRVFL-kF-Bayes, enabling all hard ks in multiple sub-learners to self-adapt to ever-changing distribution and optimization in OTCIL. Experiments were conducted on image datasets, including multiple evaluations, ablation tests, estimated forward, and compatibility studies, which distinctly validate the efficacy of edRVFL-kF-Bayes.