在最大概率方法中的性能保证规范化:Kullback-Leibler分歧中的测距对称性
1Department of Applied Mathematics, Faculty of Science, Fukuoka University, 8-19-1, Nanakuma, Jonan-ku, Fukuoka City 814-0180, Japan.
Physical review. E
|November 18, 2023
概括
本研究引入了一种用于最大概率估计的新型规范化方法,其灵感来自错误纠正代码和尺度对称. 它在没有超参数调整的情况下实现最佳概率模型,解决了数据分析中的过度拟合问题.
科学领域:
- 统计 统计 统计 统计
- 信息理论 信息理论
- 机器学习 机器学习
背景情况:
- 最大概率估计 (MLE) 是一种标准方法,用于从数据中估计概率模型.
- 传统的MLE可以通过创建太接近实证分布的模型导致过度拟合.
- 规范化方法旨在防止过度装配,但它们的系统性表现尚不清楚.
研究的目的:
- 为最大概率估计提出一个理论上有保证的规范化方法.
- 为了利用Kullback-Leibler分歧中的尺寸对称性来提高模型性能.
- 为了消除在规范化中经常进行超参数搜索的需要.
主要方法:
- 该研究在规范化和纠错代码之间进行了并行,特别是尺寸对称在最佳解码中的作用.
- 为MLE开发了一种新的规范化技术,通过将尺寸对称的原则应用于Kullback-Leibler分歧.
- 拟议的方法整合了尺寸对称性,以实现最佳的模型选择.
主要成果:
- 开发的规范化方法为最佳模型选择提供了理论保障.
- 这种方法成功地防止了过拟合,而不依赖于经验分布拟合.
- 该方法消除了超参数调整的必要性,这是规范化的常见挑战.
结论:
- 拟议的基于尺寸对称的规范化为传统方法提供了原则和有效的替代方案.
- 这种方法提高了通过MLE.估计的概率模型的稳定性和可靠性.
- 消除超参数搜索简化了统计建模中的规范化应用.
相关概念视频
One-Compartment Open Model: Wagner-Nelson and Loo Riegelman Method for ka Estimation
526
This lesson introduces two critical methods in pharmacokinetics, the Wagner-Nelson and Loo-Riegelman methods, used for estimating the absorption rate constant (ka) for drugs administered via non-intravenous routes. The Wagner-Nelson method relates ka to the plasma concentration derived from the slope of a semilog percent unabsorbed time plot. However, it is limited to drugs with one-compartment kinetics and can be impacted by factors like gastrointestinal motility or enzymatic degradation.
On...
On...
526
Propagation of Uncertainty from Random Error
699
An experiment often consists of more than a single step. In this case, measurements at each step give rise to uncertainty. Because the measurements occur in successive steps, the uncertainty in one step necessarily contributes to that in the subsequent step. As we perform statistical analysis on these types of experiments, we must learn to account for the propagation of uncertainty from one step to the next. The propagation of uncertainty depends on the type of arithmetic operation performed on...
699
Routh-Hurwitz Criterion II
257
In the application of the Routh-Hurwitz criterion, two specific scenarios can arise that complicate stability analysis.
The first scenario occurs when a singular zero appears in the first column of the Routh table. This situation creates a division by zero issues. To resolve this, a small positive or negative number, denoted as epsilon (∈), is substituted for the zero. The stability analysis proceeds by assuming a sign for ∈. If ∈ is positive, any sign change in the first...
The first scenario occurs when a singular zero appears in the first column of the Routh table. This situation creates a division by zero issues. To resolve this, a small positive or negative number, denoted as epsilon (∈), is substituted for the zero. The stability analysis proceeds by assuming a sign for ∈. If ∈ is positive, any sign change in the first...
257
Propagation of Uncertainty from Systematic Error
529
The atomic mass of an element varies due to the relative ratio of its isotopes. A sample's relative proportion of oxygen isotopes influences its average atomic mass. For instance, if we were to measure the atomic mass of oxygen from a sample, the mass would be a weighted average of the isotopic masses of oxygen in that sample. Since a single sample is not likely to perfectly reflect the true atomic mass of oxygen for all the molecules of oxygen on Earth, the mass we obtain from this...
529
Expected Frequencies in Goodness-of-Fit Tests
2.5K
A goodness-of-fit test is conducted to determine whether the observed frequency values are statistically similar to the frequencies expected for the dataset. Suppose the expected frequencies for a dataset are equal such as when predicting the frequency of any number appearing when casting a die. In that case, the expected frequency is the ratio of the total number of observations (n) to the number of categories (k).
2.5K
Routh-Hurwitz Criterion I
252
Consider an electrical power grid, where stability is essential to prevent blackouts. The Routh-Hurwitz criterion is a valuable tool for assessing system stability under varying load conditions or faults. By analyzing the closed-loop transfer function, the Routh-Hurwitz criterion helps determine whether the system remains stable.
To apply the Routh-Hurwitz criterion, a Routh table is constructed. The table's rows are labeled with powers of the complex frequency variable s, starting from the...
To apply the Routh-Hurwitz criterion, a Routh table is constructed. The table's rows are labeled with powers of the complex frequency variable s, starting from the...
252


