Generalization of stochastic-resonance-based threshold networks with Tikhonov regularization

Area of Science:

• Machine Learning
• Artificial Intelligence
• Computational Neuroscience

Background:

• Feedforward threshold neural networks traditionally lack gradient-based trainability.
• Parameter space and synaptic weight ranges are often limited in standard threshold networks.

Purpose of the Study:

• To investigate the impact of injecting artificial noise into feedforward threshold neural networks.
• To develop a stochastic-resonance-based threshold neural network trainable by gradient-based methods.
• To enhance the generalization capabilities of threshold neural networks.

Main Methods:

• Introduced artificial noise into a feedforward threshold neural network architecture.
• Developed an adaptive mechanism for noise level convergence to an optimal value.
• Provided theoretical proof for noise acting as a generalized Tikhonov regularizer.
• Conducted experiments on regression and classification tasks.

Main Results:

• The noise injection enabled gradient-based training of the threshold neural network.
• The noise level adaptively converged to a nonzero optimal value.
• Theoretical analysis confirmed noise's role as a generalized Tikhonov regularizer.
• Experimental results showed improved generalization for regression and classification problems.

Conclusions:

• Injecting noise into threshold neural networks enhances their trainability and generalization.
• The proposed stochastic-resonance-based network offers adaptive noise optimization.
• This work demonstrates the potential of adaptive stochastic resonance in machine learning applications.