Non-smooth Bayesian learning for artificial neural networks.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Optimization

Background:

• Artificial neural networks (ANNs) are crucial for signal and image analysis.
• Optimizing network weights, especially with non-smooth regularizers like the L1 norm for sparsity, presents significant challenges due to non-differentiability.

Purpose of the Study:

• To propose a novel Markov Chain Monte Carlo (MCMC)-based optimization scheme within a Bayesian framework.
• To address the challenges of optimizing sparse artificial neural networks (ANNs) efficiently.

Main Methods:

• Formulation of an MCMC-based optimization scheme in a Bayesian framework.
• Utilizing an efficient sampling scheme and Hamiltonian dynamics to solve sparse optimization problems.
• Comparative study on four datasets against two Convolutional Neural Networks (CNNs).

Main Results:

• The proposed method achieves high accuracy rates (up to $$) even with low-complexity ANNs.
• Demonstrated faster training times compared to all competing algorithms.
• Showed increased robustness against overfitting issues.

Conclusions:

• The MCMC-based Bayesian optimization scheme is effective for training sparse ANNs.
• The method offers a faster, more robust, and accurate alternative for ANN optimization.
• This approach enhances the practical applicability of ANNs in various data analysis tasks.