Learning Rates of Regularized Regression With Multiple Gaussian Kernels for Multi-Task Learning.

Area of Science:

• Machine Learning
• Statistical Learning Theory

Background:

• Multi-task learning (MTL) aims to improve generalization by learning multiple related tasks simultaneously.
• Reproducing Kernel Hilbert Spaces (RKHS) provide a powerful framework for non-linear function approximation.
• Gaussian kernels are widely used in kernel methods due to their desirable properties.

Purpose of the Study:

• To develop a least squares regularized regression algorithm for multi-task learning in a union of RKHSs with Gaussian kernels.
• To effectively utilize related task samples for selecting the unknown Gaussian kernel width for the target task.
• To achieve a fast learning rate for the target task by leveraging information from related tasks.

Main Methods:

• A least squares regularized regression algorithm is proposed.
• The algorithm operates within a union of RKHSs equipped with Gaussian kernels.
• Related task samples are used to estimate the Gaussian kernel width, which is then applied to the target task.

Main Results:

• A fast learning rate is obtained for the target task, supported by an error decomposition result.
• The algorithm demonstrates significant improvements in prediction error, especially when target task data is scarce and related task data is abundant.
• Experimental validation on simulated and real datasets confirms the algorithm's utility.

Conclusions:

• The proposed algorithm effectively enhances prediction performance in multi-task learning scenarios.
• Leveraging related tasks to determine kernel parameters is a viable strategy for improving target task learning.
• The method shows practical applicability and significant benefits in data-scarce target task situations.