Kernel-Based Multilayer Extreme Learning Machines for Representation Learning.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Deep Learning

Background:

• Multilayer extreme learning machine (ML-ELM) applied to stacked autoencoder (SAE) offers reduced training times for representation learning.
• Traditional ML-ELM faces challenges including manual hyperparameter tuning, suboptimal generalization due to random projections, and significant reconstruction errors.

Purpose of the Study:

• To introduce a kernel-based extension of ML-ELM, termed multilayer kernel ELM (ML-KELM).
• To address the limitations of ML-ELM, focusing on improved generalization, reduced reconstruction error, and efficient storage.

Main Methods:

• Developed ML-KELM inspired by kernel learning principles.
• Eliminated the need for manual tuning of hidden nodes.
• Replaced random projections with a mechanism ensuring optimal generalization.
• Utilized an exact inverse solution for output weights with invertible kernel matrices.
• Unified transformation matrices to reduce storage and execution time.

Main Results:

• ML-KELM removes the need for manual tuning of hidden nodes.
• Achieved optimal model generalization without random projection.
• Demonstrated reduced reconstruction error due to exact inverse solutions.
• Showcased significant reductions in storage requirements and execution time.
• Reported accuracy improvements of up to 7% on benchmark datasets.

Conclusions:

• ML-KELM effectively overcomes the drawbacks of ML-ELM in representation learning.
• The proposed method offers enhanced accuracy, generalization, and computational efficiency.
• ML-KELM represents a significant advancement in kernel-based machine learning techniques.