Efficient construction of sparse radial basis function neural networks using L₁-regularization.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Computational Science

Background:

• Radial Basis Function Neural Networks (RBFNNs) are effective for classification but can be computationally intensive.
• Constructing sparse RBFNNs is crucial for improving efficiency and generalization.
• Existing methods may require extensive parameter tuning or lack automatic optimization.

Purpose of the Study:

• To propose an efficient two-phase algorithm (TPCLR1) for constructing sparse Radial Basis Function Neural Networks (RBFNNs) for classification.
• To enhance model sparsity, improve generalization performance, and reduce computational resource requirements.
• To develop an automatic construction procedure requiring minimal user-defined parameters.

Main Methods:

• A two-phase construction algorithm (TPCLR1) utilizing L1 regularization.
• Phase 1: Improved Maximum Data Coverage (IMDC) for RBF center and width initialization.
• Phase 2: Specialized Orthant-Wise Limited-memory Quasi-Newton (sOWL-QN) for simultaneous network pruning and parameter optimization.

Main Results:

• TPCLR1 achieves higher model sparsity, leading to better generalization performance.
• Significant reduction in storage space and testing time compared to non-sparse methods.
• Automatic construction procedure requires only the regularization parameter and maximum function evaluations, simplifying hyperparameter tuning.
• Experimental results on classification benchmarks demonstrate ease of finding appropriate regularization parameters without cross-validation.

Conclusions:

• The proposed TPCLR1 algorithm offers an efficient and automatic method for constructing sparse RBFNN classifiers.
• The approach yields improved generalization performance and reduced computational overhead.
• TPCLR1 simplifies the RBFNN construction process, making it more accessible and practical for various classification tasks.