Sparse Bayesian extreme learning machine for multi-classification.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Computational Neuroscience

Background:

• Extreme Learning Machine (ELM) is a popular neural network with low computational cost but suffers from overfitting and sensitivity to hidden neuron count.
• Traditional ELM methods use Moore-Penrose generalized inverse for output weights, leading to least squares minimization issues and potentially large models.
• Addressing these limitations is crucial for improving ELM's practical applicability in classification tasks.

Purpose of the Study:

• To introduce a Sparse Bayesian Extreme Learning Machine (SBELM) model for classification.
• To resolve the overfitting and hidden neuron sensitivity drawbacks of standard ELM.
• To develop an accurate and compact ELM model through a novel sparse Bayesian approach.

Main Methods:

• Developed a sparse Bayesian approach for learning ELM output weights.
• Implemented marginal likelihood estimation of network outputs.
• Integrated automatic pruning of redundant hidden neurons during the learning phase.

Main Results:

• The proposed SBELM model effectively addresses overfitting issues common in ELM.
• SBELM demonstrates reduced sensitivity to the number of hidden neurons, leading to more robust performance.
• Evaluations on benchmark classification problems confirm that SBELM consistently produces more compact models compared to other state-of-the-art neural networks.

Conclusions:

• SBELM offers a significant improvement over traditional ELM by enhancing accuracy and reducing model size.
• The sparse Bayesian framework enables automatic feature selection and model compression.
• SBELM presents a promising alternative for efficient and effective classification in machine learning.