rKAN: Rational Kolmogorov-Arnold networks

Area of Science:

• Artificial Intelligence
• Deep Learning
• Machine Learning

Background:

• Kolmogorov-Arnold networks (KANs) offer an alternative to traditional multi-layer perceptrons.
• Initial KANs utilized B-spline curves, presenting implementation complexities.
• Research has explored various basis functions, including wavelets and polynomials, to enhance KANs.

Purpose of the Study:

• To introduce rational functions as a novel basis for Kolmogorov-Arnold networks (KANs).
• To develop and evaluate the rational KAN (rKAN) architecture.
• To demonstrate the efficacy of rKANs in diverse deep learning applications.

Main Methods:

• Proposed two approaches for rKANs using Padé approximation and rational Jacobi functions as trainable basis functions.
• Evaluated rKAN performance on benchmark datasets for classification and sentiment analysis.
• Assessed rKANs in a physics-informed deep learning context, specifically for reinforcement learning (CartPole).

Main Results:

• Achieved 99.29% accuracy on MNIST classification tasks.
• Attained 86.6% accuracy in text sentiment analysis.
• Successfully solved the CartPole reinforcement learning problem in approximately 200 episodes, demonstrating efficient learning.

Conclusions:

• Rational functions provide a viable and effective alternative basis for KANs.
• The proposed rKAN architecture exhibits superior performance across multiple deep learning domains.
• rKANs represent a promising advancement in neural network architectures, offering improved accuracy and efficiency.