Parsimonious extreme learning machine using recursive orthogonal least squares.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Computational Science

Background:

• Single hidden-layer feedforward networks (SLFNs) are widely used but can suffer from structural complexity and poor generalization.
• Existing extreme learning machine (ELM) methods may not always yield parsimonious models.
• Efficient structure determination and accurate prediction are crucial for SLFNs.

Purpose of the Study:

• To introduce novel constructive and destructive parsimonious extreme learning machines (CP-ELM and DP-ELM).
• To achieve parsimonious structure and excellent generalization for multi-input-multi-output SLFNs.
• To demonstrate the effectiveness of the proposed methods in nonlinear time-series modeling.

Main Methods:

• Developed CP-ELM and DP-ELM by decomposing recursive orthogonal least squares into sequential partial orthogonalization (SPO).
• Utilized random hidden node generation and recursive orthogonalization into an upper triangular matrix.
• Employed simplified termination criteria using residual error reduction and backward substitution for output weights.

Main Results:

• CP-ELM and DP-ELM achieved parsimonious architectures and superior generalization accuracy on regression datasets.
• The methods demonstrated significant reduction in matrix size during orthogonalization.
• Innovative applications to nonlinear time-series modeling yielded superior identification results.

Conclusions:

• The proposed CP-ELM and DP-ELM effectively enhance the parsimony and generalization of SLFNs.
• Sequential partial orthogonalization offers an efficient approach for model selection in ELMs.
• These novel ELM variants show promise for complex regression and time-series modeling tasks.