A Forward Learning Algorithm for Neural Memory Ordinary Differential Equations.

Area of Science:

• Artificial Intelligence
• Computational Neuroscience
• Machine Learning

Background:

• Deep neural networks using backpropagation are successful but biologically implausible.
• Existing solutions for biological plausibility are limited to discrete neural network structures.
• Continuous neural networks offer dynamic characteristics and interpretability for large language models.

Purpose of the Study:

• Introduce a novel forward-learning algorithm, nmForwardLA, for the neural memory ordinary differential equation (nmODE) model.
• Address the biological implausibility of current learning algorithms in neural networks.
• Enhance the efficiency and computational performance of continuous neural network models.

Main Methods:

• Developed a forward-learning algorithm named nmForwardLA specifically for nmODE continuous neural networks.
• Focused on reducing computational dimensions and increasing algorithmic efficiency.
• Evaluated the algorithm's performance against existing learning methods.

Main Results:

• The nmForwardLA algorithm demonstrated lower computational dimensions and improved efficiency.
• Experimental results on benchmark datasets (MNIST, CIFAR10, CIFAR100) confirmed the algorithm's effectiveness.
• The proposed method shows significant potential compared to other learning algorithms for nmODEs.

Conclusions:

• nmForwardLA provides a computationally efficient and potent learning algorithm for continuous neural networks (nmODEs).
• This biologically plausible approach advances research in dynamic neural network models and large language model interpretability.
• The algorithm's performance on standard datasets validates its practical applicability.