KKANs:库尔科瓦-科尔莫戈罗夫-阿诺德网络及其学习动态
Juan Diego Toscano1, Li-Lian Wang2, George Em Karniadakis1
1Division of Applied Mathematics, Brown University, Providence, 02912, RI, USA.
概括
与MLP和KAN相比,新的库尔科瓦-科尔莫戈罗夫-阿诺德网络 (KKAN) 提供了优越的函数近似和操作者学习. 它还在物理知情机器学习 (PIML) 中表现出强的表现,并具有用于改进概括的新注意力机制.
科学领域:
- 机器学习 机器学习
- 数字分析 数字分析
- 科学计算科学计算
背景情况:
- 科尔莫戈罗夫-阿诺德表示定理为神经网络的普遍性提供了理论基础.
- 像库尔科瓦这样的近似表示原则对于开发高效的机器学习模型至关重要.
- 像多层感知器 (MLP) 和科尔摩戈罗夫-阿诺德网络 (KAN) 这样的现有架构在某些科学应用中存在局限性.
研究的目的:
- 介绍库尔科瓦-科尔莫戈罗夫-阿诺德网络 (KKAN),一种新的神经网络架构.
- 为了证明KKAN作为通用近似仪的能力.
- 评估KKAN在各种科学机器学习任务中的表现.
主要方法:
- 开发了一个双块架构,将基于MLP的内部函数和基本函数的线性组合与外部函数相结合.
- 证明了KKAN架构的通用近似属性.
- 在函数回归,PIML和操作员学习任务上对MLP和KAN进行基准KKAN.
- 使用信息瓶理论分析学习动态和几何复杂性.
- 引入了基于剩余的自我缩放的注意力权重.
主要成果:
- KKAN在函数近似和操作员学习方面表现优于MLP和KAN.
- 在PIML任务中,KKAN实现了与优化MLP相提并论的性能.
- 确定了三个普遍的学习阶段:适应,过渡和扩散.
- 发现几何复杂性和信号噪声比 (SNR) 之间存在强烈的相关性,在扩散阶段进行最佳概括.
- 建议的注意力机制可以动态地保持高的SNR,确保统一的趋同.
结论:
- KKAN代表了用于科学机器学习的多功能和强大的新架构.
- 该研究提供了关于神经网络的学习动态和概括能力的见解.
- 提出的注意力机制可以提高模型的稳定性和在各种任务中的性能.
相关概念视频
Observational Learning
317
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
317
Introduction to Learning
537
Learning is the process of acquiring knowledge or skills through practice or experience, leading to long-lasting behavioral changes. This acquisition occurs through interaction with the environment and requires practice or experience. For instance, mastering a skill such as surfing requires considerable practice and experience, highlighting the essential role of repeated interactions with the environment in learning.
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
537
Associative Learning
593
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
593
Sequence Networks of Rotating Machines
147
A Y-connected synchronous generator, grounded through a neutral impedance, is designed to produce balanced internal phase voltages with only positive-sequence components. The generator's sequence networks include a source voltage that is exclusively in the positive-sequence network. The sequence components of line-to-ground voltages at the generator terminals illustrate this configuration.
Zero-sequence current induces a voltage drop across the generator's neutral impedance and other...
Zero-sequence current induces a voltage drop across the generator's neutral impedance and other...
147
Multi-input and Multi-variable systems
150
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
150
Parameters Affecting Nonlinear Elimination: Zero-Order Input, First-Order Absorption and Two-Compartment Model
126
Drugs administered through various routes can lead to nonlinear elimination, resulting in complex pharmacokinetic behaviors crucial to understanding efficacious drug dosing.
When a drug is administered through a constant intravenous infusion and eliminated via nonlinear pharmacokinetics, it follows zero-order input. For example, oral drugs undergo first-order absorption upon administration and are eliminated through nonlinear pharmacokinetics.
In the case of subcutaneously administered drugs,...
When a drug is administered through a constant intravenous infusion and eliminated via nonlinear pharmacokinetics, it follows zero-order input. For example, oral drugs undergo first-order absorption upon administration and are eliminated through nonlinear pharmacokinetics.
In the case of subcutaneously administered drugs,...
126


