Sparse Bayesian Learning Based on Collaborative Neurodynamic Optimization

Area of Science:

• Machine Learning
• Computational Neuroscience
• Optimization Theory

Background:

• Sparse Bayesian learning (SBL) regression often involves nonconvex global optimization.
• Existing majorization-minimization methods are sensitive to initial values, impacting solution quality.
• Nonconvex objective functions pose significant challenges for traditional optimization algorithms.

Purpose of the Study:

• To present a novel SBL algorithm utilizing collaborative neurodynamic optimization (CNO).
• To address the limitations of existing methods in finding global optimal solutions for SBL regression.
• To enhance the optimality and consistency of solutions in SBL applications.

Main Methods:

• Developed a collaborative neurodynamic optimization (CNO) system comprising recurrent neural networks (RNNs).
• Employed particle swarm optimization for repetitive reinitialization and information exchange among RNNs.
• Ensured iterative improvement of search performance towards global convergence.

Main Results:

• The proposed CNO-based SBL algorithm demonstrates almost sure convergence to a global optimal solution.
• Experimental results on sparse signal reconstruction show superior performance.
• Applications in partial differential equation identification confirm the method's efficacy and consistency.

Conclusions:

• The CNO-based SBL algorithm effectively overcomes the initialization dependency of traditional methods.
• The approach guarantees convergence to global optimal solutions for SBL regression.
• The method shows significant superiority and consistency in practical applications like signal reconstruction and PDE identification.