抽样一个人工神经网络的解决方案的空间
Alessandro Zambon1, Enrico M Malatesta2, Guido Tiana1
1INFN, Università degli Studi di Milano, Department of Physics, via Celoria 16, 20133 Milano, Italy.
Physical review. E
|November 18, 2025
概括
使用统计力学探索人工神经网络重量空间,可以发现不同的低能量的多重管拓. 在过度参数化的制度中出现了平坦的复杂结构,有助于采样,并为可扩展的网络方法提供了洞察力.
科学领域:
- 人工智能的人工智能
- 统计力学 统计力学
- 机器学习 机器学习
背景情况:
- 了解人工神经网络 (ANN) 的权重空间对于提高其性能和可扩展性至关重要.
- 无线网络的损失格局呈现出复杂的拓特征,这些特征影响了训练动态.
研究的目的:
- 使用统计力学工具系统地探索ANN的重量空间.
- 在不同的网络模式中研究低能分流器的拓.
- 为大型ANN开发可扩展的方法提供方法学的见解.
主要方法:
- 使用混合蒙特卡洛算法进行长时间的探索步骤.
- 采用基于杆的算法来分析连接路径.
- 使用合复制模型模拟来研究亚主导的平面区域.
- 针对不同能量级别和密度模式的单一隐藏层网络进行了集中分析.
主要成果:
- 观察到一个尖的拓在接近插值值的低能变频器.
- 发现在过度参数化的状态下,低能变频器变得完全平坦和复杂,从而促进采样.
- 通过数值分析证明了不同数据结构中损失景观特征的稳定性.
结论:
- 在插入和超参数化制度之间,ANN损失格局的拓显著不同.
- 过度参数化的网络中的平面,复杂的结构为优化提供了更容易访问的景观.
- 这些发现有助于为大规模神经网络开发更有效,更可扩展的训练方法.
相关概念视频
Sampling Theorem
1.2K
In signal processing, the analysis of continuous-time signals, denoted as x(t), often involves sampling techniques to convert these signals into discrete-time signals. This process is essential for digital representation and manipulation. A critical component in sampling is the train of impulses, characterized by the sampling interval and the sampling frequency. The relationship between these parameters and the original signal's properties dictates the success of the sampling process.
1.2K
Sampling Methods: Overview
2.1K
A sample refers to a smaller subset representative of a larger population. In analytical chemistry, studying or analyzing an entire population is often impractical or impossible. Therefore, samples are used to draw inferences and generalize the whole population. The sampling method selects individuals or items from a population to create a sample. Standard sampling methods include random, judgemental, systematic, stratified, and cluster sampling.
In analytical chemistry, the choice of...
In analytical chemistry, the choice of...
2.1K
Sampling Continuous Time Signal
663
In signal processing, a continuous-time signal can be sampled using an impulse-train sampling technique, followed by the zero-order hold method. Impulse-train sampling involves the use of a periodic impulse train, which consists of a series of delta functions spaced at regular intervals determined by the sampling period. When a continuous-time signal is multiplied by this impulse train, it generates impulses with amplitudes corresponding to the signal's values at the sampling points.
In the...
In the...
663
Sampling Methods: Sample Types
2.0K
Sampling materials are classified into three main types: solid, liquid, and gas.
Solid samples include a variety of substances, such as sediments from water bodies, soil, metals, and biological tissues. Two standard methods for extracting sediments from water bodies are grab sampling and piston coring. Grab sampling involves using a device to collect a discrete sediment sample from the bottom of a water body with minimal disturbance. Grab samples do not always represent the entire area due to...
Solid samples include a variety of substances, such as sediments from water bodies, soil, metals, and biological tissues. Two standard methods for extracting sediments from water bodies are grab sampling and piston coring. Grab sampling involves using a device to collect a discrete sediment sample from the bottom of a water body with minimal disturbance. Grab samples do not always represent the entire area due to...
2.0K
Random Sampling Method
14.1K
Sampling is a technique to select a portion (or subset) of the larger population and study that portion (the sample) to gain information about the population. Data are the result of sampling from a population. The sampling method ensures that samples are drawn without bias and accurately represent the population. Because measuring the entire population in a study is not practical, researchers use samples to represent the population of interest. Among the various sampling methods used by...
14.1K
Upsampling
569
Managing signal sampling rates is essential in digital signal processing to maintain signal integrity. A decimated signal, characterized by a reduced frequency range due to its lower sampling rate, can be upsampled by inserting zeros between each sample. This upsampling process expands the original spectrum and introduces repeated spectral replicas at intervals dictated by the new Nyquist frequency. To refine this zero-inserted sequence, it is passed through a lowpass filter with a cutoff...
569


