A regularized stochastic configuration network based on weighted mean of vectors for regression

Area of Science:

• Machine Learning
• Artificial Intelligence
• Computational Science

Background:

• Stochastic Configuration Networks (SCN) offer fast data modeling but are sensitive to parameter settings.
• Existing SCN models face challenges in prediction accuracy and convergence rate due to parameter tuning.
• Swarm intelligence algorithms provide novel optimization approaches for complex models.

Purpose of the Study:

• To develop a novel regularized SCN (RSCN-INFO) by integrating the weighted mean of vectors (INFO) algorithm.
• To optimize parameter selection and network structure for enhanced SCN performance.
• To improve the prediction accuracy and convergence rate of SCN models.

Main Methods:

• Introduced a regularization term combining ridge regression and residual error feedback into the SCN objective function.
• Employed the weighted mean of vectors (INFO) algorithm for automatic exploration of a four-dimensional parameter vector.
• Implemented INFO's three-phase optimization procedure: updating rule, vector combining, and local search.

Main Results:

• The proposed RSCN-INFO demonstrated superior performance in parameter setting and network compactness.
• Achieved faster reduction of network residual error compared to other algorithms.
• Exhibited enhanced convergence rates and prediction accuracy on benchmark datasets.

Conclusions:

• RSCN-INFO effectively optimizes SCN parameter selection and network structure.
• The integration of INFO algorithm significantly enhances SCN performance.
• The proposed method offers a promising approach for efficient and accurate data modeling.