Nonlinear Identification Using Orthogonal Forward Regression With Nested Optimal Regularization.

Area of Science:

• * Computational intelligence
• * Machine learning
• * Nonlinear system identification

Background:

• * Radial basis function (RBF) neural networks are powerful tools for nonlinear system identification.
• * Optimizing kernel widths and regularization parameters is crucial for maximizing generalization capability.
• * Existing methods like LROLS can be computationally intensive due to iterative optimization steps.

Purpose of the Study:

• * To introduce an efficient data-based modeling algorithm for nonlinear system identification using RBF neural networks.
• * To optimize multiple pairs of regularization parameters and kernel widths simultaneously within a single procedure.
• * To enhance the generalization capability of RBF models while reducing computational complexity.

Main Methods:

• * Orthogonal Forward Regression (OFR) procedure for model term selection.
• * Leave-One-Out (LOO) cross-validation for optimizing kernel widths and regularization parameters.
• * Simultaneous optimization of kernel widths and regularization parameters within the OFR framework, minimizing LOO mean square error (LOOMSE).

Main Results:

• * The proposed OFR algorithm produces very sparse RBF models with excellent generalization performance.
• * Achieved significant reduction in computational complexity compared to previous LROLS algorithms.
• * Demonstrated effectiveness in nonlinear system identification examples against Support Vector Machines, LASSO, and LROLS.

Conclusions:

• * The novel OFR algorithm offers an efficient and effective approach for nonlinear system identification with RBF neural networks.
• * Simultaneous optimization within the OFR procedure dramatically reduces computational cost.
• * The method yields sparse models with superior generalization capabilities.