ハミルトニアンのような深層の神経ネットワーク
1Institute for Advanced Study, University of Maryland, College Park, Maryland 20740, USA.
Physical review. E
|February 20, 2026
まとめ
この研究は,ランダムなニューラルネットワークをハミルトニアンとして見ており,そのエネルギー景観を分析しています. 無限幅の多層感知子には複雑な行動があり,一部の非線形は完全な複製対称性の破損を示しています.
科学分野:
- 理論的コンピュータ科学
- 統計学の力学 統計学の力学
- ディープラーニングの理論
背景:
- ディープラーニング理論における以前の作業は,しばしばランダムなパラメータを介してネットワーク出力を分析します.
- この研究では,逆のことを探求します:入力に対する固定ランダムネットワークのエネルギー景観.
研究 の 目的:
- ランダムに初期化された多層パーセプトン (MLP) のエネルギー景観を分析するために,その入力に対するハミルトン式として見ます.
- 無限幅の限界におけるほぼ地球最小値の構造を調査する.
- このフレームワーク内で異なるアクティベーション関数の動作を理解する.
主な方法:
- ランダムなMLPを入力に対するハミルトニアンとして見る.
- エントロピー (ログ容量) の正確な分析計算のためのレプリカトリックを使用します.
- ギブス分布を用いた入力重複のサドルポイント方程式の導出と解き方.
- 様々な深さやアクティベーション関数 (tanh, sin, ReLU,形状の非線形) に対する数値解.
主要な成果:
- 与えられたエネルギーにおけるエントロピーの正確な分析計算.
- 入力の重複を記述するサドルポイント方程式の導出.
- 数値的な解は,無限幅でも多様な振る舞いを明らかにします.
- 完全なレプリカの対称性破損は,シンのアクティベーション関数で観察されました.
- 浅い茶色/ReLUと深い形状のMLPで観察されたレプリカ対称性.
結論:
- ランダムなMLPは,アクティベーション機能と深さの影響を受け,豊かなエネルギー景観の行動を示します.
- 無限幅のネットワークは,複製対称性の破裂のような複雑な統計力学の現象を表示することができます.
- ハミルトン視点は,ディープラーニング理論とニューラルネットワークの景観の特性についての新しい洞察を提供します.
関連する概念動画
Reason and Intuition
The human brain processes information for decision-making using one of two routes: an intuitive system and a rational system (Epstein, 1994; popularized by Kahneman, 2011 as System 1 and System 2, respectively). The intuitive system is quick, impulsive, and operates with minimal effort, relying on emotions or habits to provide cues for what to do next, while the rational system is logical, analytical, deliberate, and methodical. Research in neuropsychology suggests that the brain can only use...
Neural Circuits
Neural circuits and neuronal pools are two of the main structures found in the nervous system. Neural circuits are networks of neurons that work together to carry out a specific task or process. They consist of interconnected neurons and glial cells, which provide structural and metabolic support.
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
Neurons as Communicators of the Brain
Neurons, the fundamental units of the brain and nervous system, function as the primary transmitters of information throughout the body. Their ability to communicate through electrical and chemical signals is vital for every bodily function, from regulating the heartbeat to processing complex thoughts. Each neuron has three main components: the cell body (soma), dendrites, and an axon, each specialized to facilitate swift and efficient neural communication.
Cell Body
The cell body, also known...
Cell Body
The cell body, also known...
Introduction to Cognitive Psychology
Cognitive psychology is the field of psychology dedicated to examining how people think. It attempts to explain how and why we think the way we do by studying the interactions among human thinking, emotion, creativity, language, and problem-solving, as well as other cognitive processes. Cognitive psychology studies how information is processed and manipulated in remembering, thinking, and knowing.
This field emerged in the mid-20th century, following a period dominated by behaviorism, which...
This field emerged in the mid-20th century, following a period dominated by behaviorism, which...
Storage
A schema is a mental framework that helps individuals organize and interpret information. Schemata, formed from previous experiences, influence how we process new information: how we encode it, the inferences we make, and how we retrieve it. For instance, a schema for what a typical classroom looks like might include desks, a teacher's desk, a whiteboard, and students in such an environment. This expectation helps us quickly understand and navigate new classrooms without needing to analyze each...


