在深度神经网络中特征学习的弹块理论
Cheng Shi1, Liming Pan2, Ivan Dokmanić1,3
1University of Basel, Departement Mathematik und Informatik, Spiegelgasse 1, 4051 Basel, Switzerland.
Physical review letters
|July 31, 2025
概括
深度神经网络通过将数据压缩成更简单的几何形状来学习特征. 一个新的阶段图和机械理论揭示了噪音和非线性如何影响跨网络层的学习有效性,将特征学习与概括联系起来.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 计算神经科学是一种神经科学.
背景情况:
- 深度神经网络 (DNN) 通过逐步减少数据维度来表现特征学习.
- 了解这种低维几何体从微观动力学中出现的原因仍然是当前理论面临的挑战.
研究的目的:
- 阐明DNN中特征学习中非线性和噪声的作用.
- 开发一个理论框架,解释特征学习如何跨网络层进行.
主要方法:
- 为DNN构建了一个噪声非线性相位图.
- 开发了一个宏观的机械理论来建模特征学习动态.
主要成果:
- 确定了不同的模式,浅层或深层基于噪音和非线性更有效地学习.
- 提出的机械理论成功地复制了观察到的相位图.
- 建立了跨层特征学习和模型概括之间的联系.
结论:
- 非线性和噪声极大地影响了DNN中特征学习的有效性.
- 宏观机械视角为深度网络中的特征学习和概括理解提供了一个统一的理论.
相关概念视频
Introduction to Learning
533
Learning is the process of acquiring knowledge or skills through practice or experience, leading to long-lasting behavioral changes. This acquisition occurs through interaction with the environment and requires practice or experience. For instance, mastering a skill such as surfing requires considerable practice and experience, highlighting the essential role of repeated interactions with the environment in learning.
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
533
Observational Learning
314
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
314
Associative Learning
579
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
579
Neural Circuits
1.6K
Neural circuits and neuronal pools are two of the main structures found in the nervous system. Neural circuits are networks of neurons that work together to carry out a specific task or process. They consist of interconnected neurons and glial cells, which provide structural and metabolic support.
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
1.6K
Cognitive Learning
526
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
526
Parallel Processing
229
The brain processes sensory information rapidly due to parallel processing, which involves sending data across multiple neural pathways at the same time. This method allows the brain to manage various sensory qualities, such as shapes, colors, movements, and locations, all concurrently. For instance, when observing a forest landscape, the brain simultaneously processes the movement of leaves, the shapes of trees, the depth between them, and the various shades of green. This enables a quick and...
229


