Bayesian Random Vector Functional-Link Networks for Robust Data Modeling.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Computational Statistics

Background:

• Random Vector Functional-Link (RVFL) networks are a type of randomized neural network.
• RVFL networks are typically trained using regularized least-squares methods for a point estimate of weights.
• Existing methods lack uncertainty quantification and the ability to incorporate prior knowledge effectively.

Purpose of the Study:

• To introduce novel Bayesian inference (BI) strategies for training RVFL networks.
• To enable the derivation of probability distributions over network weights, moving beyond point estimates.
• To enhance RVFL networks with uncertainty quantification and prior knowledge integration.

Main Methods:

• Development of two full Bayesian inference algorithms for RVFL regression, utilizing iterative processes and closed-form computations.
• Implementation of a general variational inference strategy applicable to various data modeling scenarios.
• Leveraging automatic differentiation for broad applicability of the proposed methods.

Main Results:

• The proposed Bayesian RVFL algorithm demonstrated superior performance compared to standard training methods when using a carefully selected regularization factor.
• Bayesian training provides a probability distribution over weights, offering uncertainty estimates.
• The variational inference approach is effective for data modeling with noisy outputs or outliers.

Conclusions:

• Full Bayesian inference offers significant advantages for training RVFL networks, including improved performance and uncertainty estimation.
• The proposed Bayesian training algorithms are computationally efficient and can outperform traditional methods.
• Variational inference provides a flexible framework for applying Bayesian principles to RVFL networks in diverse applications.