A comparative analysis of support vector machines and extreme learning machines

Area of Science:

• Machine Learning
• Artificial Intelligence
• Computational Neuroscience

Background:

• Extreme Learning Machines (ELMs) are increasingly popular for single-hidden-layer feed-forward neural networks.
• ELMs offer advantages like low computational cost, good generalization, and ease of implementation.
• Comparative analysis and model selection between ELMs and other machine learning approaches are significant research areas.

Purpose of the Study:

• To perform a comparative analysis of basic ELMs and Support Vector Machines (SVMs).
• To investigate the Vapnik-Chervonenkis (VC) dimension of ELMs.
• To evaluate the performance of ELMs and SVMs under varying training sample sizes.

Main Methods:

• Analysis of the Vapnik-Chervonenkis (VC) dimension for ELMs.
• Empirical evaluation of generalization ability and computational complexity.
• Comparison of ELM and SVM performance across different training data scales.

Main Results:

• The VC dimension of an ELM is equivalent to its number of hidden nodes with probability one.
• ELMs exhibit weaker generalization than SVMs with small sample sizes.
• ELMs demonstrate comparable generalization to SVMs with large sample sizes.
• ELMs show superior computational speed, particularly for large-scale problems.

Conclusions:

• ELMs provide a computationally efficient alternative to SVMs, especially for large datasets.
• Understanding the VC dimension offers theoretical insight into ELM generalization.
• The findings complement existing experimental and theoretical comparisons between ELMs and SVMs.