Translation-Invariant Kernels for Multivariable Approximation

Area of Science:

• Machine Learning
• Applied Mathematics
• Signal Processing

Background:

• Shallow neural networks with translation-invariant kernels are widely used for function approximation and classification.
• Understanding the theoretical underpinnings of these networks is crucial for optimizing their performance.

Purpose of the Study:

• To investigate the suitability of shallow networks with translation-invariant kernel units for function approximation and classification.
• To identify critical properties of kernel functions that influence network capabilities.

Main Methods:

• Analysis of the convergence behavior of Fourier transforms of kernel functions.
• Utilizing the Hankel transform to analyze multivariable kernels.
• Illustrating general results with examples of univariable and multivariable kernels (Gaussian, Laplace, rectangle, sinc, cut power).

Main Results:

• A critical property influencing kernel network capabilities is how kernel Fourier transforms converge to zero.
• Kernels for multivariable approximation require Fourier transforms that can be negative but are almost everywhere nonzero.
• Kernels for maximal margin classification require nonnegative Fourier transforms that can be zero over large sets.

Conclusions:

• The behavior of kernel Fourier transforms dictates suitability for specific tasks.
• Distinct properties of kernel Fourier transforms are necessary for effective function approximation and maximal margin classification.