通过随机条件独立的hessians进行深度失学.
Ronak Mehta1, Sourav Pal1, Vikas Singh1
1University of Wisconsin-Madison.
概括
机器取消学习从预测模型中删除数据,这对隐私和数据完整性至关重要. 一种新方法,L-CODEC,有效地识别了无需复杂的计算来消除学习的模型参数,从而使视觉和NLP中的应用成为可能.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 数据 隐私 数据 隐私 数据
背景情况:
- 由于隐私法规和数据质量问题,机器取消学习越来越重要.
- 现有的复杂模型中去学习的方法在计算上昂贵,通常需要黑西矩阵反转.
- 像k-NN这样的更简单的模型可以通过删除数据来取消学习,但这不适用于深度学习模型.
研究的目的:
- 在复杂的预测模型中开发一种高效,可扩展的机器取消学习方法.
- 为了解决现有的基于优化的取消学习技术的计算局限性.
- 为了使计算机视觉和自然语言处理等领域的深度学习模型能够进行近似的不学习.
主要方法:
- 利用L-CODEC (条件独立系数) 的一种变体来识别单个样本中具有高度语义重叠的模型参数.
- 采用马尔科夫面选择来确定有针对性的放弃学习的相关参数.
- 避免了需要计算密集的赫森矩阵反转.
主要成果:
- 通过L-CODEC方法,成功地确定了一组模型参数,以有效地消除学习.
- 该方法有效地与模型维度进行扩展,克服了以前技术的局限性.
- 对于深度视觉和NLP模型来说,已经证明了近似取消学习的可行性.
结论:
- 在深度学习模型中,L-CODEC为深度学习模型中的机器取消学习提供了一个计算可行的解决方案.
- 这种方法在敏感的应用程序中,如面部识别,人重新识别和NLP,有助于消除学习.
- 这种方法使得在以前无法实现的场景中,近似的失学变得可行.
相关概念视频
Randomized Experiments
7.0K
The randomization process involves assigning study participants randomly to experimental or control groups based on their probability of being equally assigned. Randomization is meant to eliminate selection bias and balance known and unknown confounding factors so that the control group is similar to the treatment group as much as possible. A computer program and a random number generator can be used to assign participants to groups in a way that minimizes bias.
Simple randomization
Simple...
Simple randomization
Simple...
7.0K
Avoidance Learning and Learned Helplessness
1.8K
Avoidance learning and learned helplessness are critical concepts in understanding behavioral responses to negative stimuli.
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
1.8K
Hess's Law
45.2K
There are two ways to determine the amount of heat involved in a chemical change: measure it experimentally, or calculate it from other experimentally determined enthalpy changes. Some reactions are difficult, if not impossible, to investigate and make accurate measurements for experimentally. And even when a reaction is not hard to perform or measure, it is convenient to be able to determine the heat involved in a reaction without having to perform an experiment.
45.2K
Associative Learning
450
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
450
Purposive Learning
145
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
145
Woodward–Hoffmann Selection Rules and Microscopic Reversibility
3.2K
Electrocyclic reactions, cycloadditions, and sigmatropic rearrangements are concerted pericyclic reactions that proceed via a cyclic transition state. These reactions are stereospecific and regioselective. The stereochemistry of the products depends on the symmetry characteristics of the interacting orbitals and the reaction conditions. Accordingly, pericyclic reactions are classified as either symmetry-allowed or symmetry-forbidden. Woodward and Hoffmann presented the selection criteria for...
3.2K


