Stochastic Gradient Descent for Kernel-Based Maximum Correntropy Criterion.

Area of Science:

• Machine Learning
• Signal Processing
• Optimization Theory

Background:

• Maximum Correntropy Criterion (MCC) offers robust learning against non-Gaussian noise and outliers, unlike traditional Least Squares (LS).
• MCC involves non-convex optimization, an area with less mature theoretical understanding compared to convex optimization.
• Kernel methods enhance MCC's ability to handle nonlinear structures.

Purpose of the Study:

• To rigorously analyze the convergence behavior of stochastic gradient descent (SGD) applied to the kernel Maximum Correntropy Criterion (MCC).
• To establish explicit convergence rates for kernel MCC using SGD under standard conditions.
• To bridge the theoretical gap between optimization processes and convergence guarantees in robust learning.

Main Methods:

• Application of stochastic gradient descent (SGD) algorithm to the kernel version of Maximum Correntropy Criterion (MCC).
• Rigorous mathematical analysis of the convergence properties of the applied SGD algorithm.
• Derivation of explicit convergence rates under specified theoretical conditions.

Main Results:

• Established theoretical guarantees for the convergence of SGD in kernel MCC.
• Provided explicit convergence rates, quantifying the algorithm's efficiency.
• Demonstrated that while iterates may converge to a global minimizer, the resulting estimator does not guarantee global optimality.

Conclusions:

• The study provides a foundational theoretical analysis for SGD in kernel MCC, crucial for understanding its performance in robust nonlinear modeling.
• The findings contribute to the theoretical understanding of non-convex optimization in machine learning.
• Highlights the distinction between iterate convergence and estimator optimality in the context of kernel MCC.