变压器和皮质波:编码器用于在时间跨度中拉取上下文
Lyle Muller1, Patricia S Churchland2, Terrence J Sejnowski3
1Department of Mathematics, Western University, London, Ontario, Canada; Fields Laboratory for Network Science, Fields Institute, Toronto, Ontario, Canada.
Trends in neurosciences
|September 28, 2024
概括
大脑中的皮层波可能通过编码神经活动来像变压器网络一样起作用. 这种机制可以使大脑能够处理连续的信息,类似于大型语言模型 (LLM).
科学领域:
- 计算神经科学是一种计算神经科学.
- 人工智能的人工智能是人工智能.
- 认知科学是一种认知科学.
背景情况:
- 像ChatGPT这样的大型语言模型 (LLM) 在使用变压器网络处理顺序数据方面表现出色.
- 变压器依靠自我注意机制来捕捉输入序列中的远程时间依赖.
- 了解大脑中时间信息处理的神经基础是一个关键的挑战.
研究的目的:
- 提出一种神经生物学机制来处理时间序列,类似于变压器网络.
- 探索皮层波如何可以实现类似的计算原理,作为在LLMs自我注意力.
主要方法:
- 理论建模将变压器计算与神经活动模式进行比较.
- 对自我注意机制及其潜在神经相关的概念分析.
- 假设移动波在神经信息编码中的作用.
主要成果:
- 变压器网络将输入序列编码为向量,从而能够捕获时间依赖.
- 变压器中的自我注意力计算着对联的单词关联,以增强上下文.
- 皮层波被提议将输入历史封装成空间模式,使时间上下文提取成为可能.
结论:
- 皮层波可以提供转换器网络中使用的编码原理的生物实现.
- 这种神经机制可以解释大脑如何处理连续的感官信息.
- 这项研究提供了一个新的视角,将人工智能计算与大脑功能联系起来.
相关概念视频
Types Of Transformers
951
Transformers can provide desired voltages to a circuit by modifying the number of turns in the secondary windings.
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
If the ratio of the number of turns in the secondary winding to that of the primary winding is greater than one, then the transformer is said to be a step-up transformer. In a step-up transformer, the voltage at the secondary winding is greater than the voltage applied at the primary winding.
However, if this ratio is less than one, the transformer is said to be a step-down...
951
Transformers in Distribution System
99
Transformers in distribution systems can be broadly categorized into distribution substation transformers and other distribution transformers. They are crucial for stepping down high transmission voltages to levels suitable for distribution and end-user applications.
Distribution substation transformers come in various ratings and typically use mineral oil for insulation and cooling. To prevent moisture and air from entering the oil, some transformers use an inert gas like nitrogen to fill the...
Distribution substation transformers come in various ratings and typically use mineral oil for insulation and cooling. To prevent moisture and air from entering the oil, some transformers use an inert gas like nitrogen to fill the...
99
Transformers with Off-Nominal Turns Ratios
141
In scenarios involving parallel transformers with disparate ratings, developing per-unit models requires accommodating off-nominal turns ratios. This situation arises when the selected base voltages are not proportional to the transformer’s voltage ratings. Consider a transformer where the rated voltages are related by the term a. If the chosen voltage bases satisfy a relationship involving term b, term c is defined as the ratio of these bases. This ratio is then substituted into the...
141
Brain Waves
1.1K
Brain waves are electrical signals generated by the neurons in the brain, which are regularly monitored to measure mental activities. Brain waves and their frequency ranges can be measured using an electroencephalogram or EEG. There are four main types of brain waves, each with distinct characteristics:
1.1K
The Ideal Transformer
359
In single-phase two-winding transformers, two windings are coiled around a magnetic core characterized by cross-sectional area A and magnetic permeability μ. A phasor current i1 enters the left winding while i2 exits the right winding, establishing the fundamental working of the transformer through electromagnetic principles.
Ampere's Law forms the basis of understanding the magnetic field within the transformer. It states that the integral of the magnetic field intensity's...
Ampere's Law forms the basis of understanding the magnetic field within the transformer. It states that the integral of the magnetic field intensity's...
359


