无调节的Coreset马尔科夫链蒙特卡洛通过热 DOG
Naitong Chen1, Jonathan H Huggins2, Trevor Campbell1
1Department of Statistics, University of British Columbia, Vancouver, BC, Canada.
概括
我们介绍了热启动距离与梯度 (Hot DoG),这是一种用于训练贝叶斯核心设定重量的新方法. 热DoG消除了在Coreset马尔科夫链蒙特卡洛 (MCMC) 的学习速度调整的需要,提高后方近似质量.
科学领域:
- 计算统计学 计算统计学
- 机器学习 机器学习
- 贝叶斯的推理是贝叶斯的推理.
背景情况:
- 贝叶斯核心集通过将大型数据集与较小,加权的子集进行近似来提供计算节省.
- 当前的Coreset马尔科夫链蒙特卡洛 (MCMC) 方法依赖于随机梯度优化,这种优化对学习速率超参数敏感.
- 低于最佳的学习速度可能会降低核心集的质量及其后置近似值.
研究的目的:
- 为培训贝叶斯核心设置权重开发一种没有学习率的优化程序.
- 为了提高Coreset MCMC算法中的稳定性并减少用户调整力度.
- 为了提高使用贝叶斯核心集获得的后方近似的质量.
主要方法:
- 提出热启动距离对梯度 (Hot DoG),一种新的无学习率的随机梯度优化算法.
- 提供与Hot DoG.训练的核心组重量的收性质的理论分析.
- 在经验上对现有的学习率免费方法和像ADAM这样的自适应优化器进行热狗的评估.
主要成果:
- 热DoG成功地训练了核心设置重量,而不需要手动学习速度选择.
- 理论分析证实了由Hot DoG产生的核心设置重量的趋同.
- 经验结果表明,与其他没有学习率的方法相比,Hot DoG产生了优越的后方近似,并且与调整的ADAM相竞争.
结论:
- 在Coreset MCMC中,Hot DoG提供了一个强大的,用户友好的替代方案来训练贝叶斯核心集.
- 拟议的方法可以在没有超参数调整的情况下实现高质量的后方近似.
- 这一进步降低了大数据集的贝叶斯推理计算成本和复杂性.
相关概念视频
Maxam-Gilbert Sequencing
12.5K
In the same year as the discovery of the Sanger sequencing method, another group of scientists, Allan Maxam and Walter Gilbert, demonstrated their chemical-cleavage method for DNA sequencing. The Maxam-Gilbert method relies on using different chemicals that can cleave the DNA sequence at specific sites, the separation of resulting DNA fragments of variable size using electrophoresis, and deciphering the DNA sequence from the resulting gel bands.
Challenges of the Maxam-Gilbert Method
The...
Challenges of the Maxam-Gilbert Method
The...
12.5K
Random Variables
17.2K
A random variable is a single numerical value that indicates the outcome of a procedure. The concept of random variables is fundamental to the probability theory and was introduced by a Russian mathematician, Pafnuty Chebyshev, in the mid-nineteenth century.
Uppercase letters such as X or Y denote a random variable. Lowercase letters like x or y denote the value of a random variable. If X is a random variable, then X is written in words, and x is given as a number.
For example, let X = the...
Uppercase letters such as X or Y denote a random variable. Lowercase letters like x or y denote the value of a random variable. If X is a random variable, then X is written in words, and x is given as a number.
For example, let X = the...
17.2K
Randomized Experiments
8.8K
The randomization process involves assigning study participants randomly to experimental or control groups based on their probability of being equally assigned. Randomization is meant to eliminate selection bias and balance known and unknown confounding factors so that the control group is similar to the treatment group as much as possible. A computer program and a random number generator can be used to assign participants to groups in a way that minimizes bias.
Simple randomization
Simple...
Simple randomization
Simple...
8.8K
Per-Unit Sequence Models
404
An ideal Y-Y transformer, grounded through neutral impedances, displays per-unit sequence networks akin to those of a single-phase ideal transformer when subjected to balanced positive- or negative-sequence currents. These currents do not produce neutral currents, and their associated voltage drops.
Zero-sequence currents, which are identical in magnitude and phase, generate a neutral current, resulting in voltage drops across the neutral impedance and the low-voltage winding. If the...
Zero-sequence currents, which are identical in magnitude and phase, generate a neutral current, resulting in voltage drops across the neutral impedance and the low-voltage winding. If the...
404
Sequence Networks of Rotating Machines
467
A Y-connected synchronous generator, grounded through a neutral impedance, is designed to produce balanced internal phase voltages with only positive-sequence components. The generator's sequence networks include a source voltage that is exclusively in the positive-sequence network. The sequence components of line-to-ground voltages at the generator terminals illustrate this configuration.
Zero-sequence current induces a voltage drop across the generator's neutral impedance and other...
Zero-sequence current induces a voltage drop across the generator's neutral impedance and other...
467
One-Compartment Open Model: Wagner-Nelson and Loo Riegelman Method for ka Estimation
1.1K
This lesson introduces two critical methods in pharmacokinetics, the Wagner-Nelson and Loo-Riegelman methods, used for estimating the absorption rate constant (ka) for drugs administered via non-intravenous routes. The Wagner-Nelson method relates ka to the plasma concentration derived from the slope of a semilog percent unabsorbed time plot. However, it is limited to drugs with one-compartment kinetics and can be impacted by factors like gastrointestinal motility or enzymatic degradation.
On...
On...
1.1K


