Hierarchical-Bayesian-Based Sparse Stochastic Configuration Networks for Construction of Prediction Intervals

Area of Science:

• Machine Learning
• Artificial Intelligence
• Computational Statistics

Background:

• Neural networks face challenges with high-dimensional data due to architecture complexity and ill-posed problems.
• Existing methods often struggle with efficient training and reliable uncertainty quantification.

Purpose of the Study:

• To develop a novel Bayesian-learning-based sparse stochastic configuration network (BSSCN) for enhanced performance on high-dimensional data.
• To improve the training efficiency and analytical solvability of output weights.
• To construct accurate prediction intervals (PIs) by accounting for data noise and model mismatch.

Main Methods:

• The BSSCN utilizes a Laplace distribution as the prior for output weights, replacing the common Gaussian distribution.
• A two-level hierarchical prior approximates a sparse Gaussian posterior, facilitating analytical solutions for output weights.
• Hyperparameter estimation is performed using the expectation-maximization algorithm by maximizing the lower bound of the marginal likelihood.
• A bootstrap ensemble strategy is employed to generate prediction intervals.

Main Results:

• The proposed BSSCN demonstrates effectiveness on benchmark and real-world high-dimensional datasets.
• Experimental results show significant improvements in prediction accuracy compared to existing methods.
• The method successfully constructs high-quality prediction intervals, effectively capturing uncertainties.

Conclusions:

• The BSSCN offers an effective solution for handling high-dimensional data in machine learning.
• The Bayesian learning approach with Laplace priors enhances model sparsity and training efficiency.
• The bootstrap ensemble strategy provides reliable uncertainty quantification through prediction intervals.