通过机器取消学习来缓解LLMs4Code中的敏感信息泄露
Shanzhi Gu1, Zhaoyang Qu1, Ruotong Geng2
1College of Computer Science and Technology, National University of Defense Technology, Changsha, 410073, Hunan, China.
概括
机器取消学习显著减少了大型语言代码模型 (LLMs4Code) 中的敏感数据泄漏,将直接泄漏减少了50%以上,同时保留了91%的编码能力. 需要进一步的研究来解决剩余的间接泄漏问题.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 软件工程 软件工程 软件工程
背景情况:
- 大型代码语言模型 (LLMs4Code) 在代码生成方面表现出色,但有可能泄露敏感的训练数据.
- 现有的LLMs4Code隐私措施不足以防止敏感信息的披露.
研究的目的:
- 本研究提供了第一个全面的实证分析机器取消学习的缓解敏感数据泄露在LLMs4Code.
- 评估机器取消学习在降低隐私风险的有效性,同时保持模型性能.
主要方法:
- 用合成"忘记"和"保留"数据集创建了一个专门的基准,以测试隐私和功能.
- 在三个开源LLMs4Code模型 (AIXCoder-7B,CodeLlama-7B,CodeQwen-7B) 上,对三个机器取消学习算法 (GA,GA+GD,GA+KL) 进行了系统评估.
主要成果:
- 机器取消学习可以平均减少50%以上的直接数据泄露.
- 在解除学习后,代码生成性能保持在91%以上.
- 从直接到间接的数据泄露的转变被观察到在学习停止后,这表明持续的脆弱性.
结论:
- 机器取消学习是提高LLMs4Code隐私的可行和有效方法.
- 未来的研究必须开发技术,同时解决直接和间接泄漏问题,以获得强大的隐私保护.
相关概念视频
Machines
563
Machines are complex structures consisting of movable, pin-connected multi-force members that work together to transmit forces. One example of a machine is the cutting plier, which is used to cut wires by applying forces to its handles. When equal and opposite forces are exerted on the handles of the cutting plier, they cause the cutting edges to come together and apply equal and opposite reaction forces on the wire, which are greater than the applied forces.
A free-body diagram of the...
A free-body diagram of the...
563
Machines: Problem Solving II
652
Machines are complex structures consisting of movable, pin-connected multi-force members that work together to transmit forces. Consider a lifting tong carrying a 100 kg load. It comprises movable sections DAF and CBG linked together with member AB.
652
Machines: Problem Solving I
701
A toggle clamp is a mechanical device commonly used for holding and clamping objects in various applications, such as woodworking, metalworking, and assembly operations. Consider a toggle clamp subjected to a force of 200 N at the handle. The vertical clamping force can be calculated, provided the dimensions of the toggle clamp are known.
The toggle clamp system is a machine structure consisting of movable, pin-connected multi-force members that form a stabilized system to transmit forces. The...
The toggle clamp system is a machine structure consisting of movable, pin-connected multi-force members that form a stabilized system to transmit forces. The...
701
Sequence Networks of Rotating Machines
489
A Y-connected synchronous generator, grounded through a neutral impedance, is designed to produce balanced internal phase voltages with only positive-sequence components. The generator's sequence networks include a source voltage that is exclusively in the positive-sequence network. The sequence components of line-to-ground voltages at the generator terminals illustrate this configuration.
Zero-sequence current induces a voltage drop across the generator's neutral impedance and other...
Zero-sequence current induces a voltage drop across the generator's neutral impedance and other...
489
Simplified Synchronous Machine Model
759
The Synchronous Machine Model is a fundamental tool in analyzing and ensuring the transient stability of power systems. This model simplifies the representation of a synchronous machine under balanced three-phase positive-sequence conditions, assuming constant excitation and ignoring losses and saturation. The model is pivotal for understanding the behavior of synchronous generators connected to a power grid, particularly during transient events.
In this model, each generator is connected to a...
In this model, each generator is connected to a...
759
Wind Turbine Machine Models
576
In the growing field of wind energy, incorporating wind turbine models into transient stability analysis is essential. Induction and synchronous machines are the primary models used, with induction machines being prevalent due to their simplicity and reliability.
Induction machines interact through the rotating magnetic field generated by the stator and the rotor. The key parameter is slip, which is the difference between synchronous speed and rotor speed relative to synchronous speed. Slip is...
Induction machines interact through the rotating magnetic field generated by the stator and the rotor. The key parameter is slip, which is the difference between synchronous speed and rotor speed relative to synchronous speed. Slip is...
576


