Scalable learning method for feedforward neural networks using minimal-enclosing-ball approximation.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Computational Neuroscience

Background:

• Training feedforward neural networks (FNNs) is computationally intensive and space-consuming for large datasets.
• Existing FNN training methods struggle with scalability due to high complexity.
• Addressing these limitations is crucial for advancing FNN applications.

Purpose of the Study:

• To develop a novel, efficient learning algorithm for FNNs applicable to very large datasets.
• To address the Center-Constrained Minimum Enclosing Ball (CCMEB) problem in FNN hidden feature space.
• To reduce the computational and space complexity of FNN training.

Main Methods:

• A novel learning algorithm, HFSR-GCVM (hidden-feature-space regression using generalized core vector machine), is proposed.
• A new learning criterion using an L2-norm penalty-based ε-insensitive function is formulated.
• Random generation of hidden node parameters, independent of training sets, is employed.

Main Results:

• The learning of output layer parameters is equivalent to a special CCMEB problem in FNN hidden feature space.
• HFSR-GCVM exhibits maximal training time linear with dataset size.
• Maximal space consumption is independent of dataset size.

Conclusions:

• HFSR-GCVM effectively trains FNNs on large datasets with improved efficiency.
• The algorithm overcomes the computational and space complexity limitations of traditional FNN training.
• Experimental results on regression tasks validate the proposed method's effectiveness and scalability.