A Comparative Study of Pairwise Learning Methods Based on Kernel Ridge Regression.

Area of Science:

• Machine Learning
• Statistical Learning Theory

Background:

• Pairwise learning, also known as dyadic prediction or network inference, is crucial for many machine learning tasks.
• Kernel methods have dominated pairwise learning, achieving state-of-the-art performance.
• Theoretical analysis of kernel methods in pairwise learning remains underexplored.

Purpose of the Study:

• To review and unify kernel-based algorithms used in various pairwise learning settings.
• To provide a theoretical analysis of these methods, focusing on Kronecker kernel ridge regression.
• To offer insights into the advantages and limitations of existing pairwise learning techniques.

Main Methods:

• Focus on closed-form, efficient instantiations of Kronecker kernel ridge regression.
• Unification of independent task kernel ridge regression, two-step kernel ridge regression, and linear matrix filters.
• Analysis of universality, consistency, and spectral filtering properties.

Main Results:

• Demonstration that independent task kernel ridge regression, two-step kernel ridge regression, and linear matrix filters are special cases of Kronecker kernel ridge regression.
• Establishment that these unified methods implicitly minimize a squared loss.
• Theoretical insights into the behavior and properties of kernel-based pairwise learning algorithms.

Conclusions:

• The study provides a unified theoretical framework for kernel-based pairwise learning methods.
• Understanding the implicit squared loss minimization offers valuable insights for method selection and development.
• This work bridges the gap between practical application and theoretical understanding in pairwise learning.