Parsimonious kernel extreme learning machine in primal via Cholesky factorization

Area of Science:

• Machine Learning
• Kernel Methods

Background:

• Extreme Learning Machine (ELM) is a popular machine learning technique.
• Kernel ELM (KELM) methods, P-KELM and D-KELM, offer advantages but suffer from dense solutions proportional to training data size.

Purpose of the Study:

• To develop novel algorithms that address the computational inefficiency of dense solutions in P-KELM and D-KELM.
• To improve the training speed and reduce the cost of kernel ELM methods.

Main Methods:

• Proposed a constructive algorithm for P-KELM (CCP-KELM) using Cholesky factorization to select significant vectors.
• Introduced PCCP-KELM by incorporating a probabilistic speedup scheme into CCP-KELM.
• Developed a destructive P-KELM (CDP-KELM) utilizing partial Cholesky factorization to prune non-significant data.

Main Results:

• The proposed CCP-KELM and CDP-KELM algorithms effectively reduce the number of significant vectors.
• Experiments on benchmark datasets demonstrate the efficacy and feasibility of the new algorithms.
• PCCP-KELM further enhances training cost reduction through a probabilistic speedup.

Conclusions:

• CCP-KELM and CDP-KELM offer efficient alternatives to traditional dense KELM methods.
• The developed algorithms provide a practical solution for large-scale machine learning tasks.
• The study validates the effectiveness of selective data vector selection in kernel ELM algorithms.