Is extreme learning machine feasible? A theoretical assessment (part I)

Area of Science:

• Machine Learning
• Artificial Intelligence
• Computational Neuroscience

Background:

• Extreme Learning Machines (ELMs) are feedforward neural networks (FNNs) with randomly fixed hidden layer connections.
• ELMs have shown high efficiency in many applications, but their general feasibility remains an open question.
• This study provides a comprehensive feasibility analysis of ELMs.

Purpose of the Study:

• To theoretically justify the generalization capability of ELMs for suitable activation functions.
• To determine the number of hidden neurons required for ELMs to achieve theoretical bounds.
• To investigate the impact of different activation functions on ELM performance and generalization.

Main Methods:

• Theoretical analysis of ELM generalization bounds.
• Examination of activation functions including polynomials, Nadaraya-Watson, and sigmoid functions.
• Application of generalized inverse techniques and Tikhonov regularization for solving ELM systems.

Main Results:

• ELMs with suitable activation functions can achieve the theoretical generalization bound of FNNs.
• Methods for estimating the required number of hidden neurons were developed.
• The study identified specific activation functions that can degrade ELM generalization capability.
• Efficient solution techniques for ELMs were proposed, including generalized inverse and Tikhonov regularization.

Conclusions:

• ELMs are feasible and efficient for a range of applications, particularly with specific activation functions.
• The choice of activation function is critical for maintaining ELM generalization capability.
• The findings offer insights for improving and generalizing ELM systems.