SineKAN: Kolmogorov-Arnold Networks using sinusoidal activation functions.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Kolmogorov-Arnold Networks (KAN) offer an alternative to traditional multi-layer perceptron neural networks.
• KANs utilize learnable activation functions on graph edges and summation on nodes.
• Original KAN implementations employ basis spline (B-Spline) functions for edge activations.

Purpose of the Study:

• Introduce SineKAN, a KAN variant using re-weighted sine functions instead of B-Splines.
• Evaluate SineKAN's numerical performance and scalability on a benchmark vision task.
• Compare SineKAN against B-Spline KAN and Fourier Series KAN models.

Main Methods:

• Replaced B-Spline activation functions with grids of re-weighted sine functions in the KAN framework.
• Assessed model performance on a standard computer vision benchmark.
• Benchmarked SineKAN against B-Spline KAN and a Fourier Series KAN implementation.

Main Results:

• SineKAN demonstrated performance comparable or superior to B-Spline KAN and Fourier Series KAN.
• Achieved significant speed increases across various hidden layer sizes, batch sizes, and network depths.
• Showed numerical accuracy potentially scalable to the level of dense neural networks (DNNs).

Conclusions:

• SineKAN presents a computationally efficient and accurate alternative within the KAN framework.
• The model offers substantial speed advantages over existing KAN implementations.
• Further research is warranted to explore SineKAN's theoretical scaling and address current limitations compared to optimized DNNs.