LeanKAN: a parameter-lean Kolmogorov-Arnold network layer with improved memory efficiency and convergence behavior

Area of Science:

• Machine Learning
• Artificial Intelligence
• Deep Learning

Background:

• Kolmogorov-Arnold Networks (KANs) present a novel architecture as an alternative to Multi-Layer Perceptrons (MLPs) for data-driven modeling.
• The MultKAN layer was introduced to enhance KANs by incorporating both addition and multiplication subnodes, aiming for improved representational power.

Purpose of the Study:

• To identify and address the limitations of MultKAN layers, specifically their restricted use in output layers, complex parameterization, and numerous hyperparameters.
• To introduce LeanKANs as a direct, modular, and improved replacement for both MultKAN and traditional AddKAN layers.

Main Methods:

• Proposed LeanKANs as a layer replacement, designed for general applicability in output layers, reduced parameter counts, and simplified hyperparameter sets.
• Evaluated LeanKANs through direct layer replacement in standard KAN tasks and augmented structures like KAN Ordinary Differential Equations (KAN-ODEs) and Deep Operator KANs (DeepOKANs).

Main Results:

• LeanKANs demonstrate superior performance compared to MultKANs, even when the latter have larger parameter counts, across various tasks including differential equations.
• The sparser parameterization and compact structure of LeanKANs enhance their expressivity and learning capabilities.

Conclusions:

• LeanKANs provide a more efficient and effective alternative to MultKAN and AddKAN layers, offering significant advantages in parameter efficiency and performance.
• LeanKANs are versatile, serving as a backbone for advanced KAN architectures and improving learning outcomes in complex problems like KAN-ODEs.