神经网络中的编码方案学习分类任务
Alexander van Meegen1, Haim Sompolinsky2,3
1Center for Brain Science, Harvard University, Cambridge, MA, 02138, USA. alexander.vanmeegen@epfl.ch.
Nature communications
|April 9, 2025
概括
神经网络学习任务特定的特征,但它们的表示取决于神经元的非线性. 线性网络使用模拟编码,而非线性网络由于对称性破坏而呈现稀疏或冗余的编码.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 计算神经科学是一种神经科学.
背景情况:
- 神经网络擅长学习任务依赖的特征.
- 这些新出现的表征的精确性质仍然不太清楚.
- 了解功能学习是推动人工智能和神经科学发展的关键.
研究的目的:
- 研究学习如何在广泛,完全连接的神经网络中塑造表征.
- 分析神经元非线性对新出现的特征表示的影响.
- 探索贝叶斯框架,以了解神经网络重量后期.
主要方法:
- 利用贝叶斯框架来建模神经网络重量的后部分布.
- 分析了完全连接的,广泛的神经网络,训练了分类任务.
- 专注于网络运营的特征学习 ("非") 模式.
主要成果:
- 网络获得了强大的数据依赖特征 (编码方案),其中响应与类成员关系相关.
- 编码方案的类型严重取决于神经元的非线性.
- 线性网络开发模拟编码方案; 非线性网络通过对称性破坏展示稀疏或冗余编码.
结论:
- 神经网络的非线性是决定神经网络中新出现的表征性质的关键因素.
- 像权重缩放和非线性这样的网络属性显著塑造了学习到的表示.
- 这些发现提供了对人工神经系统和生物神经系统特征学习机制的见解.
相关概念视频
Classification of Systems-I
161
Linearity is a system property characterized by a direct input-output relationship, combining homogeneity and additivity.
Homogeneity dictates that if an input x(t) is multiplied by a constant c, the output y(t) is multiplied by the same constant. Mathematically, this is expressed as:
Homogeneity dictates that if an input x(t) is multiplied by a constant c, the output y(t) is multiplied by the same constant. Mathematically, this is expressed as:
161
Classification of Systems-II
127
Continuous-time systems have continuous input and output signals, with time measured continuously. These systems are generally defined by differential or algebraic equations. For instance, in an RC circuit, the relationship between input and output voltage is expressed through a differential equation derived from Ohm's law and the capacitor relation,
127
Classification of Signals
348
In signal processing, signals are classified based on various characteristics: continuous-time versus discrete-time, periodic versus aperiodic, analog versus digital, and causal versus noncausal. Each category highlights distinct properties crucial for understanding and manipulating signals.
A continuous-time signal holds a value at every instant in time, representing information seamlessly. In contrast, a discrete-time signal holds values only at specific moments, often denoted as x(n), where...
A continuous-time signal holds a value at every instant in time, representing information seamlessly. In contrast, a discrete-time signal holds values only at specific moments, often denoted as x(n), where...
348
Neural Circuits
942
Neural circuits and neuronal pools are two of the main structures found in the nervous system. Neural circuits are networks of neurons that work together to carry out a specific task or process. They consist of interconnected neurons and glial cells, which provide structural and metabolic support.
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
942
Associative Learning
253
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
253
Aggregates Classification
291
Aggregate classification is generally based on its size, petrographic characteristics, weight, and source. Size classification ranges from coarse to fine aggregates, defined by the size of the particles. Coarse aggregates are particles that do not pass through ASTM sieve No. 4, and aggregates that pass through the sieve are fine aggregates.
Petrographic classification groups aggregates based on common mineralogical characteristics. Some of the common mineral groups found in aggregates are...
Petrographic classification groups aggregates based on common mineralogical characteristics. Some of the common mineral groups found in aggregates are...
291


