重尾更新分布源于非平衡学习中的信息驱动的自我组织
Xin-Ya Zhang1,2, Chao Tang1
1Center for Interdisciplinary Studies and Department of Physics, School of Science, Westlake University, Hangzhou 310030, People's Republic of China.
概括
人工神经网络在训练过程中表现出自我组织的关键性,平衡探索和适应. 这种由信息原则驱动的动态过程揭示了人工智能学习和可解释性的洞察力.
科学领域:
- 人工智能的人工智能
- 计算神经科学是一种神经科学.
- 机器学习 机器学习
背景情况:
- 人工神经网络 (ANN) 经常在参数空间中的探索与特定任务的适应之间取得平衡,反映人类决策.
- 在培训期间了解ANN的动态对于提高AI性能和可解释性至关重要.
研究的目的:
- 在神经网络训练期间识别一致的关键性标志.
- 为信息驱动的自我组织提供理论证据,作为观察到缩放行为背后的机制.
- 调查损失景观的内在几何特性和学习过程的性质.
主要方法:
- 在神经网络训练期间识别一致的关键性标志.
- 开发基于最大和相互信息原则的理论模型.
- 数字模拟来证明自我组织的关键性,并分析损失的景观特性.
- 在参数更新和更新间隔中分析功率定律分布.
主要成果:
- 在神经网络训练过程中发现了一致的关键性标志.
- 理论证据表明,关键性来自最大 (探索) 和相互信息 (任务相关性) 之间的平衡.
- 损失景观显示了从指数的过渡到强度规律的性,表明了内在的几何性质.
- 更新间隔中的权力规律分布表明间歇性学习过程.
结论:
- 神经网络的学习是一个不平衡的过程,由随机性和相关性之间的权衡决定.
- 自组织的批判性为理解ANN的动态性和适应性提供了一个框架.
- 这些发现为人工智能系统的解释性提供了洞察力.
相关概念视频
Probability Distributions
11.6K
The probability of a random variable x is the likelihood of its occurrence. A probability distribution represents the probabilities of a random variable using a formula, graph, or table. There are two types of probability distribution– discrete probability distribution and continuous probability distribution.
A discrete probability distribution is a probability distribution of discrete random variables. It can be categorized into binomial probability distribution and Poisson...
A discrete probability distribution is a probability distribution of discrete random variables. It can be categorized into binomial probability distribution and Poisson...
11.6K
Survival Tree
362
Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
Building a Survival Tree
Constructing a...
Building a Survival Tree
Constructing a...
362
Genetic Drift
42.8K
Natural selection—probably the most well-known evolutionary mechanism—increases the prevalence of traits that enhance survival and reproduction. However, evolution does not merely propagate favorable traits, nor does it always benefit populations.
42.8K
Distributed Loads: Problem Solving
1.0K
Beams are structural elements commonly employed in engineering applications requiring different load-carrying capacities. The first step in analyzing a beam under a distributed load is to simplify the problem by dividing the load into smaller regions, which allows one to consider each region separately and calculate the magnitude of the equivalent resultant load acting on each portion of the beam. The magnitude of the equivalent resultant load for each region can be determined by calculating...
1.0K
Choosing Between z and t Distribution
3.5K
The z and the Student t distribution estimate the population mean using the sample mean and standard deviation. However, to decide which distribution to use for a calculation, one needs to determine the sample size, the nature of the distribution, and whether the population standard deviation is known. If the population standard deviation is known and the population is normally distributed, or if the sample size is greater than 30, the z distribution is preferred. The Student t distribution is...
3.5K
Distribution Reliability and Automation
469
Distribution reliability in electrical power systems is critical for ensuring an uninterrupted power supply to consumers at minimal cost. According to IEEE Standard Terms, reliability is the probability that a device will function without failure over a specified time period or amount of usage. For electric power distribution, this translates to maintaining continuous power supply and addressing customer concerns over power outages. Several indices, as defined by IEEE Standard 1366-2012, are...
469


