通过潜伏的相互信息揭示神经学习动态
Arianna Issitt1, Alex Merino1, Lamine Deen1
1NEural TransmissionS (NETS) Lab, Florida Institute of Technology, Melbourne, FL 32901, USA.
Entropy (Basel, Switzerland)
|January 28, 2026
概括
卷积神经网络在学习过程中将信息集中在特定的道中,而不是普遍压缩信息. 这种选择性集中提高了准确性,加快了训练的速度.
科学领域:
- 计算机科学 计算机科学
- 机器学习 机器学习
- 人工智能的人工智能
背景情况:
- 卷积神经网络 (CNN) 是图像分类的关键.
- 在学习过程中,了解CNN内部的信息流动至关重要.
研究的目的:
- 研究CNN在自然图像分类过程中如何重组信息.
- 分析相互信息 (MI) 在CNN表示中的作用.
主要方法:
- 在VGG-16,ResNet-18和ResNet-50中跟踪输入,中间表示和标签之间的相互信息.
- 分析特定通道中的信息度,使用淘汰,混合和扰动.
- 基于希尔伯特 - 施密特独立标准的依赖意识调节器的实施.
主要成果:
- 标签相关的MI随网络深度而增加,而输入MI则随架构而变化.
- 信息集中在一组通道中,这些通道在功能上是准确性所必需的.
- 一个促进选择性度的调节器提高了收率和精度.
结论:
- CNN的表示学习是由选择性集中和脱关系驱动的,而不是全球压缩.
- 有针对性的规范化可以引导CNN实现高效的信息处理.
- 研究结果为设计更有效的深度学习模型提供了洞察力.
相关概念视频
Mutual Inductance
3.8K
Inductance is the property of a device that tells us how effectively it induces an emf in another device. In other words, it is a physical quantity that expresses the effectiveness of a given device.
When two circuits carrying time-varying currents are close to one another, the magnetic flux through each circuit varies because of the changing current in the other circuit. Consequently, an emf is induced in each circuit by the changing current in the other. Therefore, this type of emf is called...
When two circuits carrying time-varying currents are close to one another, the magnetic flux through each circuit varies because of the changing current in the other circuit. Consequently, an emf is induced in each circuit by the changing current in the other. Therefore, this type of emf is called...
3.8K
Avoidance Learning and Learned Helplessness
2.6K
Avoidance learning and learned helplessness are critical concepts in understanding behavioral responses to negative stimuli.
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
2.6K
Neural Regulation
43.3K
Digestion begins with a cephalic phase that prepares the digestive system to receive food. When our brain processes visual or olfactory information about food, it triggers impulses in the cranial nerves innervating the salivary glands and stomach to prepare for food.
43.3K
Dynamic Equilibrium
62.4K
A reversible chemical reaction represents a chemical process that proceeds in both forward (left to right) and reverse (right to left) directions. When the rates of the forward and reverse reactions are equal, the concentrations of the reactant and product species remain constant over time and the system is at equilibrium. A special double arrow is used to emphasize the reversible nature of the reaction. The relative concentrations of reactants and products in equilibrium systems vary greatly;...
62.4K
Associative Learning
1.3K
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
1.3K
Purposive Learning
470
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
470


