相关实验视频
Updated: May 14, 2025
07:46
Setting Limits on Supersymmetry Using Simplified Models
Published on: November 15, 2013
8.5K
在过度参数化的制度中,免费的隐私
Simone Bombari1, Marco Mondelli1
1Institute of Science and Technology Austria, Klosterneuburg 3400, Austria.
概括
不同的私有梯度下降 (DP-GD) 为深度学习模型提供了隐私. 在过度参数化的环境中,DP-GD可以在没有性能成本的情况下实现隐私,挑战现有的信念.
科学领域:
- 机器学习 机器学习
- 深度学习 (Deep Learning) 是一种深度学习.
- 保护隐私的技术 保护隐私的技术
背景情况:
- 差分私有梯度下降 (DP-GD) 对于训练深度学习模型至关重要,同时保护数据隐私.
- 现有的研究已经确定了与标准梯度下降 (GD) 相比,DP-GD的性能成本的上限.
- 这些界限在过度参数化的模式中往往会变得更糟,这在深度学习中很常见,为从业者创造了不确定性.
研究的目的:
- 在过度参数化的深度学习模型中研究DP-GD的性能成本.
- 挑战传统的理解,即过度参数化本质上降低了保护隐私的学习表现.
- 为实践者在大型模型中导航DP-GD提供理论见解.
主要方法:
- 在随机特征模型中的分析具有二次性损失.
- 在不同的隐私参数下对DP-GD性能进行理论检查.
- 调查过度参数化 (与样本相对的参数数量大) 对DP-GD的影响.
主要成果:
- 证明,对于足够大的模型,可以在没有过度人口风险的情况下实现隐私 ([公式:见文本]).
- 这一发现甚至适用于常量级隐私参数 ([公式:见文本]) 和强烈私密设置 ([公式:见文本]).
- 结果表明,过度参数化不一定会阻碍差异化私人学习的表现.
结论:
- 深度学习中的过度参数化在使用DP-GD时本质上不会给性能带来处罚.
- 与常见假设相反,在某些过度参数化的场景中,隐私可以"免费"获得.
- 这项工作提供了新的理论指导,表明大型模型可以与强有力的隐私保证相兼容.
相关概念视频
01:13Randomized Experiments
6.6K
The randomization process involves assigning study participants randomly to experimental or control groups based on their probability of being equally assigned. Randomization is meant to eliminate selection bias and balance known and unknown confounding factors so that the control group is similar to the treatment group as much as possible. A computer program and a random number generator can be used to assign participants to groups in a way that minimizes bias.
Simple randomization
Simple...
Simple randomization
Simple...
6.6K
00:59Propagation of Uncertainty from Random Error
577
An experiment often consists of more than a single step. In this case, measurements at each step give rise to uncertainty. Because the measurements occur in successive steps, the uncertainty in one step necessarily contributes to that in the subsequent step. As we perform statistical analysis on these types of experiments, we must learn to account for the propagation of uncertainty from one step to the next. The propagation of uncertainty depends on the type of arithmetic operation performed on...
577
01:09Censoring Survival Data
47
Survival analysis is a statistical method used to analyze time-to-event data, often employed in fields such as medicine, engineering, and social sciences. One of the key challenges in survival analysis is dealing with incomplete data, a phenomenon known as "censoring." Censoring occurs when the event of interest (such as death, relapse, or system failure) has not occurred for some individuals by the end of the study period or is otherwise unobservable, and it might have many different...
47
01:34Woodward–Hoffmann Selection Rules and Microscopic Reversibility
3.0K
Electrocyclic reactions, cycloadditions, and sigmatropic rearrangements are concerted pericyclic reactions that proceed via a cyclic transition state. These reactions are stereospecific and regioselective. The stereochemistry of the products depends on the symmetry characteristics of the interacting orbitals and the reaction conditions. Accordingly, pericyclic reactions are classified as either symmetry-allowed or symmetry-forbidden. Woodward and Hoffmann presented the selection criteria for...
3.0K
01:10Propagation of Uncertainty from Systematic Error
412
The atomic mass of an element varies due to the relative ratio of its isotopes. A sample's relative proportion of oxygen isotopes influences its average atomic mass. For instance, if we were to measure the atomic mass of oxygen from a sample, the mass would be a weighted average of the isotopic masses of oxygen in that sample. Since a single sample is not likely to perfectly reflect the true atomic mass of oxygen for all the molecules of oxygen on Earth, the mass we obtain from this...
412
01:19Routh-Hurwitz Criterion II
161
In the application of the Routh-Hurwitz criterion, two specific scenarios can arise that complicate stability analysis.
The first scenario occurs when a singular zero appears in the first column of the Routh table. This situation creates a division by zero issues. To resolve this, a small positive or negative number, denoted as epsilon (∈), is substituted for the zero. The stability analysis proceeds by assuming a sign for ∈. If ∈ is positive, any sign change in the first...
The first scenario occurs when a singular zero appears in the first column of the Routh table. This situation creates a division by zero issues. To resolve this, a small positive or negative number, denoted as epsilon (∈), is substituted for the zero. The stability analysis proceeds by assuming a sign for ∈. If ∈ is positive, any sign change in the first...
161