Direct Kernel Perceptron (DKP): ultra-fast kernel ELM-based classification with non-iterative closed-form weight...

Area of Science:

• Machine Learning
• Computational Science
• Pattern Recognition

Background:

• Kernel-based classifiers like Support Vector Machines (SVM) and Extreme Learning Machines (ELM) are widely used but can be computationally intensive.
• Existing methods often require iterative training or tunable regularization parameters, impacting efficiency and ease of use.

Purpose of the Study:

• To introduce the Direct Kernel Perceptron (DKP), a novel, fast, and efficient kernel-based classifier.
• To demonstrate that DKP offers a competitive trade-off between classification accuracy and computational speed compared to established algorithms.

Main Methods:

• The DKP utilizes a Gaussian kernel and a linear classifier, calculating coefficients directly via a closed-form expression without iterative training.
• It minimizes an error measure combining training error and hyperplane margin, avoiding tunable regularization parameters.
• The linear DKP is presented as a special case of the two-class Extreme Learning Machine (ELM) with specific regularization parameter values (C=0).

Main Results:

• The linear DKP significantly outperforms 12 popular classifiers in terms of speed across 42 benchmark datasets.
• DKP achieves higher accuracy than 7 of these classifiers and shows comparable results to slower, top-performing methods like SVM and ELM.
• The DKP provides a superior balance of accuracy and efficiency, particularly with the linear error function.

Conclusions:

• The Direct Kernel Perceptron (DKP) offers a computationally efficient and accurate classification method.
• DKP presents a viable alternative to existing kernel methods, especially when speed and simplicity are critical.
• Freely available C and Matlab code facilitate the adoption and further research of DKP.