Fast Deep Stacked Networks based on Extreme Learning Machine applied to regression problems.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Deep learning excels with large datasets but suffers from long training times.
• Stacked network architectures offer improved efficiency but face challenges with training time and memory.
• Fast Deep Stacked Network (FDSN) uses Extreme Learning Machine (ELM) variants to address these issues.

Purpose of the Study:

• To investigate the impact of weight initialization on Fast Deep Stacked Network (FDSN) performance.
• To propose FKDSN, a kernel-based variant of FDSN.
• To analyze the theoretical complexity of FDSN and FKDSN.

Main Methods:

• Evaluated three weight initialization methods for ELM-trained neural networks.
• Tested methods on 50 public real-world regression datasets.
• Introduced FKDSN, a kernel-based approach.

Main Results:

• Optimized FDSN with complex initialization matched large SLFN performance with reduced training time and memory.
• FKDSN demonstrated comparable results to large SLFNs, with lower memory requirements.
• The proposed methods are suitable for resource-limited systems like IoT devices.

Conclusions:

• Weight initialization is crucial for optimizing FDSN performance.
• FDSN and FKDSN offer efficient alternatives to large SLFNs, especially for edge computing.
• These advancements facilitate deep learning deployment on devices with limited computational resources.