Synaptic plasticity-based regularizer for artificial neural networks.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computational Neuroscience

Background:

• Artificial neural networks (ANNs) require regularization for generalization.
• Existing methods lack constraints for real-world deployment with changing input data distributions.
• Neuroplasticity offers inspiration for robust model adaptation.

Purpose of the Study:

• To introduce a bounded regularization method for ANNs applicable during deployment.
• To enhance the adaptability and robustness of ANNs in variable environments.
• To improve model performance on classification and regression tasks.

Main Methods:

• Extended neuronal masking to generate supporting neurons, improving output reliability.
• Incorporated a synaptic connection module with online optimization via synaptic rewiring.
• Formulated the optimization as bilevel mixed-integer nonlinear programming (MINLP) solved with a single-wave scheme.
• Proposed a storage/recovery memory module for efficient knowledge retrieval.

Main Results:

• Achieved approximately 8% improvement in accuracy on classification and regression tasks compared to state-of-the-art methods.
• Demonstrated enhanced adaptability and robustness of ANN models in dynamic environments.
• Validated the effectiveness of the bounded regularization approach during deployment.

Conclusions:

• The proposed bounded regularization method significantly improves ANN performance and reliability.
• The neuroplasticity-inspired approach offers a robust solution for ANNs facing input data distribution shifts.
• This technique holds promise for deploying ANNs in real-world, unpredictable environments.