Selective Memory Recursive Least Squares: Recast Forgetting Into Memory in RBF Neural Network-Based Real-Time...

Area of Science:

• Artificial Intelligence
• Machine Learning
• Neural Networks

Background:

• Radial Basis Function Neural Networks (RBFNNs) utilize forgetting mechanisms for real-time learning to maintain sensitivity to new data.
• Classical forgetting mechanisms can lead to the loss of valuable, older information, termed passive knowledge forgetting.
• Existing methods like Forgetting Factor Recursive Least Squares (FFRLS) and Stochastic Gradient Descent (SGD) have limitations in retaining historical knowledge.

Purpose of the Study:

• To introduce a novel real-time training method, Selective Memory Recursive Least Squares (SMRLS), for RBFNNs.
• To overcome the passive knowledge forgetting phenomenon inherent in traditional forgetting mechanisms.
• To enhance the learning speed and generalization capabilities of RBFNNs in dynamic environments.

Main Methods:

• Recasting classical forgetting mechanisms into a memory mechanism that evaluates sample importance based on temporal and spatial distributions.
• Dividing the RBFNN input space into partitions and developing a synthesized objective function using samples from each partition.
• Updating neural network weights using both current approximation error and recorded data from visited partitions.

Main Results:

• SMRLS demonstrates improved learning speed compared to FFRLS and SGD.
• The proposed SMRLS method exhibits enhanced generalization capability.
• Simulation results validate the effectiveness of SMRLS in mitigating passive knowledge forgetting.

Conclusions:

• SMRLS offers an effective solution to passive knowledge forgetting in RBFNNs.
• The method's dual consideration of temporal and spatial data distributions enhances knowledge retention.
• SMRLS represents a significant advancement for real-time learning in RBFNN applications.