Robust C-Loss Kernel Classifiers.

Area of Science:

• Machine Learning
• Robust Statistics
• Kernel Methods

Background:

• Outliers can significantly degrade the performance of standard machine learning classifiers.
• Correntropy-induced loss (C-loss) offers inherent robustness to outliers.
• Tikhonov regularization is crucial for preventing overfitting in kernel methods.

Purpose of the Study:

• To investigate the properties and performance of a C-loss kernel classifier with Tikhonov regularization.
• To establish the equivalence between the proposed C-loss classifier and iterative weighted LS-SVM.
• To enhance the efficiency and sparseness of the C-loss kernel classifier for large-scale datasets.

Main Methods:

• Utilized the half-quadratic optimization algorithm for faster convergence compared to gradient-based methods.
• Established theoretical equivalence between C-loss kernel classifier and iterative weighted LS-SVM.
• Applied incomplete Cholesky decomposition for efficient training on large datasets with low-rank Gram matrices.
• Leveraged the representer theorem to induce sparseness in the classifier.

Main Results:

• The C-loss kernel classifier was shown to be equivalent to iterative weighted LS-SVM.
• This equivalence provides insights into the robustness of iterative weighted LS-SVM from correntropy and density estimation viewpoints.
• Incomplete Cholesky decomposition significantly speeds up training for large-scale datasets.
• The representer theorem effectively improved the sparseness of the classifier.
• Experimental results demonstrated superior robustness against outliers compared to existing classifiers.

Conclusions:

• The proposed C-loss kernel classifier offers significant robustness to outliers.
• The method provides computational advantages through faster training and improved sparseness.
• The findings contribute to a deeper understanding of robust kernel methods and their practical applications.