Optimal Randomness for Stochastic Configuration Network (SCN) with Heavy-Tailed Distributions

Area of Science:

• Machine Learning
• Artificial Intelligence
• Neural Networks

Background:

• Determining optimal neural network architecture is challenging for performance.
• Stochastic Configuration Networks (SCN) offer a randomized approach to network training.
• Original SCN uses uniform distribution for random weight and bias initialization.

Purpose of the Study:

• To investigate the impact of heavy-tailed distributions on SCN performance.
• To explore if heavy-tailed initializations can improve upon uniform distribution in SCN.
• To enhance regression and classification capabilities of SCN models.

Main Methods:

• Implemented SCN with heavy-tailed distributions (Lévy, Cauchy, Weibull) for random initialization.
• Utilized mixed Gaussian and Laplace distributions exhibiting heavy-tailed properties.
• Compared performance metrics of new SCN variants against the original SCN.

Main Results:

• SCN models with heavy-tailed distributions demonstrated improved performance.
• For regression, SCN variants required fewer hidden nodes for comparable results.
• Classification models using SCN-Mixture, SCN-Lévy, and SCN-Cauchy achieved higher test accuracies (up to 92.4%).

Conclusions:

• Heavy-tailed distributions offer superior randomness for SCN initialization.
• The proposed SCN variants outperform the original SCN in regression and classification.
• This research provides a novel method for optimizing SCN architecture and performance.