Scaling Up Generalized Kernel Methods.

Area of Science:

• Machine Learning
• Artificial Intelligence

Background:

• Kernel methods are successful but struggle with big data scalability.
• Existing methods face limitations in training and prediction efficiency.

Purpose of the Study:

• To develop a scalable sparse kernel learning formulation for big data.
• To introduce an efficient algorithm for large-scale kernel learning.

Main Methods:

• Introduced a general sparse kernel learning formulation using random feature approximation (possibly non-convex loss).
• Utilized orthogonal random feature approximation to reduce feature scale.
• Proposed a novel asynchronous parallel doubly stochastic algorithm (AsyDSSKL).

Main Results:

• AsyDSSKL is the first algorithm combining asynchronous parallel computation and doubly stochastic optimization.
• Comprehensive convergence guarantees were provided for AsyDSSKL.
• Experimental results demonstrated significant computational efficiency superiority over existing kernel methods on large-scale datasets.

Conclusions:

• AsyDSSKL effectively addresses the scalability limitations of traditional kernel methods.
• The proposed method offers superior computational efficiency for large-scale sparse kernel learning.
• AsyDSSKL represents a significant advancement in handling big data with kernel methods.