SGB-ELM: An Advanced Stochastic Gradient Boosting-Based Ensemble Scheme for Extreme Learning Machine.

Area of Science:

• Machine Learning
• Artificial Intelligence

Background:

• Extreme Learning Machine (ELM) is a popular single-hidden layer feedforward network.
• Ensemble methods are widely used to improve ELM performance.
• Overfitting remains a challenge in ELM and its ensemble variants.

Purpose of the Study:

• To propose a novel ensemble scheme for Extreme Learning Machine (ELM) called Stochastic Gradient Boosting-based Extreme Learning Machine (SGB-ELM).
• To enhance the generalization performance and predictive robustness of ELM models.

Main Methods:

• Developed SGB-ELM by constructing a sequence of weak ELMs trained additively.
• Designed a regularized objective function based on a boosting mechanism to mitigate overfitting.
• Utilized second-order optimization to derive output-layer weights, incorporating a heuristic initialization and iterative updates.

Main Results:

• SGB-ELM demonstrated superior generalization performance compared to existing ELM ensemble methods.
• The proposed method showed improved predictive robustness.
• The feasibility and effectiveness of SGB-ELM were validated through comparative analysis.

Conclusions:

• SGB-ELM offers an effective approach for improving ELM ensemble performance.
• The integration of stochastic gradient boosting with regularization addresses overfitting challenges.
• This novel method provides a robust alternative for machine learning applications requiring high generalization.