SineKAN:使用正弦激活函数的科尔摩戈罗夫-阿诺德网络
Eric Reinhardt1, Dinesh Ramakrishnan1, Sergei Gleyzer1
1Department of Physics and Astronomy, The University of Alabama, Tuscaloosa, AL, United States.
Frontiers in artificial intelligence
|January 30, 2025
概括
新型神经网络SineKAN用重新加权的正弦函数取代了基线函数. 这一创新为密集神经网络提供了可比的准确性,并比现有的科尔摩戈罗夫-阿诺德网络 (KAN) 显著提高了速度.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 深度学习 (Deep Learning) 是一种深度学习.
背景情况:
- 科尔摩戈罗夫-阿诺德网络 (KAN) 为传统的多层感知神经网络提供了替代方案.
- KAN在图边上使用可学习的激活函数,在节点上使用总和函数.
- 原始的KAN实现采用了边缘激活的基础分线 (B-Spline) 函数.
研究的目的:
- 介绍SineKAN,一种使用重量化正弦函数而不是B-Splines的KAN变体.
- 在一个基准视觉任务上评估SineKAN的数值性能和可扩展性.
- 将SineKAN与B-Spline KAN和福利埃系列KAN模型进行比较.
主要方法:
- 在KAN框架中,用重新加权的正弦函数的网格取代了B-Spline激活函数.
- 在标准计算机视觉基准上评估模型性能.
- 比较的SineKAN与B-Spline KAN和富里埃序列KAN实现.
主要成果:
- 斯尼肯的性能与B-Spline KAN和福利埃系列 KAN相比或优于它们.
- 在各种隐藏层大小,批量大小和网络深度中实现了显著的速度增加.
- 显示了数值准确性,可能可扩展到密集神经网络 (DNN) 的水平.
结论:
- 在KAN框架内,SineKAN提供了一个计算效率高,准确的替代方案.
- 该模型比现有的KAN实现提供了显著的速度优势.
- 需要进一步的研究来探索SineKAN的理论扩展,并解决与优化DNN相比目前的局限性.
相关概念视频
Graphical and Analytic Representation of Sinusoids
369
Analyzing two sinusoidal voltages with equal amplitude and period but different phases on an oscilloscope, an instrument used to display and analyze waveforms, involves a three-step process.
The first step is measuring the peak-to-peak value, which is twice the amplitude of the sinusoid. This provides information about the maximum voltage swing of the waveform.
Secondly, the period and angular frequency are determined. The period is the time taken for one complete cycle of the waveform, while...
The first step is measuring the peak-to-peak value, which is twice the amplitude of the sinusoid. This provides information about the maximum voltage swing of the waveform.
Secondly, the period and angular frequency are determined. The period is the time taken for one complete cycle of the waveform, while...
369
Exponential and Sinusoidal Signals
223
The exponential function is crucial for characterizing waveforms that rise and decay rapidly. This continuous-time exponential function is defined using exponential terms with constants α and A. When both constants are real, the function is represented as,
223
Sinusoidal Sources
467
Direct current (DC) refers to an electric current that flows in a single direction, maintaining a constant polarity. This is in contrast to alternating current (AC), which periodically changes its direction and magnitude. AC forms the backbone of modern electricity transmission and distribution systems due to its efficient long-distance transmission capabilities.
In homes, the power supplies use sinusoidal sources to provide electricity. These sources generate a voltage that varies sinusoidally...
In homes, the power supplies use sinusoidal sources to provide electricity. These sources generate a voltage that varies sinusoidally...
467
Network Function of a Circuit
255
Frequency response analysis in electrical circuits provides vital insights into a circuit's behavior as the frequency of the input signal changes. The transfer function, a mathematical tool, is instrumental in understanding this behavior. It defines the relationship between phasor output and input and comes in four types: voltage gain, current gain, transfer impedance, and transfer admittance. The critical components of the transfer function are the poles and zeros.
255
SFG Algebra
107
In Signal Flow Graph (SFG) algebra, the value a node represents is determined by the sum of all signals entering that node. This summed value is then transmitted through every branch leaving the node, making the SFG a powerful tool for visualizing and analyzing control systems.
Each node in an SFG corresponds to a variable, and the interactions between nodes are represented by branches with associated gains. When multiple branches lead into a node, the value at that node is the sum of the...
Each node in an SFG corresponds to a variable, and the interactions between nodes are represented by branches with associated gains. When multiple branches lead into a node, the value at that node is the sum of the...
107
Propagation of Action Potentials
5.1K
The propagation of an action potential refers to the process by which a nerve impulse, or "action potential," travels along a neuron.
Neurons (nerve cells) have a resting membrane potential, with a slightly negative charge inside compared to outside. This is maintained by ion channels, such as sodium (Na+) and potassium (K+) channels, which control the flow of ions. When a stimulus, like a touch or a signal from another neuron, triggers the neuron, sodium channels open, allowing sodium ions to...
Neurons (nerve cells) have a resting membrane potential, with a slightly negative charge inside compared to outside. This is maintained by ion channels, such as sodium (Na+) and potassium (K+) channels, which control the flow of ions. When a stimulus, like a touch or a signal from another neuron, triggers the neuron, sodium channels open, allowing sodium ions to...
5.1K


