Distributed learning for sketched kernel regression.

Area of Science:

• Machine Learning
• Statistical Learning Theory
• Distributed Computing

Background:

• Distributed learning addresses large datasets by dividing them into subsets.
• The divide-and-conquer strategy averages local estimators from data subsets.
• Existing methods may still involve large subsets, necessitating further computational reduction.

Purpose of the Study:

• To investigate the integration of random sketching with divide-and-conquer for distributed regularized least squares regression.
• To enhance computational efficiency in distributed learning settings.
• To analyze the theoretical performance of this combined approach.

Main Methods:

• Distributed regularized least squares regression in a reproducing kernel Hilbert space (RKHS).
• Application of random sketching to generate local estimators on data subsets.
• Theoretical analysis of learning rates under the combined strategy.
• Comparative simulations of sketched and non-sketched divide-and-conquer methods.

Main Results:

• Demonstration that random sketching and divide-and-conquer are complementary for large sample sizes.
• Theoretical proof that optimal learning rates can be achieved.
• Empirical validation through simulations comparing the methods.

Conclusions:

• The combination of random sketching and divide-and-conquer offers an effective strategy for distributed learning.
• This approach significantly reduces computational load while preserving learning performance.
• The findings are applicable to large-scale machine learning problems in RKHS.