选择性知识共享,以保护隐私,在没有良好的教师的情况下进行联合蒸
Jiawei Shao1, Fangzhao Wu2, Jun Zhang3
1Hong Kong University of Science and Technology, Hong Kong, China.
Nature communications
|January 8, 2024
概括
联合蒸 (FD) 通过分享知识而不是数据来改善协作学习. 选择性FD通过识别精确的知识,提高绩效,并实现保护隐私,高效和适应性的联合培训来增强这一点.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 分布式系统 分布式系统
背景情况:
- 联合学习 (FL) 提供保护隐私的协作培训,但面临着隐私攻击,通信开销和模型异质性等挑战.
- 联合蒸 (FD) 通过转移知识而不是模型参数来解决FL的局限性,但由于数据变化和缺乏教师模型,它遭受了模两可的知识共享.
研究的目的:
- 为联邦蒸 (FD) 提出一种新的选择性知识共享机制,以提高其性能和稳定性.
- 加强联合培训框架的隐私保护,沟通效率和异质性适应能力.
主要方法:
- 引入了选择性FD,这是FD框架内从局部和整体预测中识别准确和精确知识的机制.
- 进行实证研究并提供理论见解以验证拟议的方法.
主要成果:
- 选择性FD显著提高了联合蒸框架的泛化能力.
- 拟议的方法在联合学习场景中始终优于现有的基线方法.
结论:
- 选择性FD有效地解决了FD中模两可的知识共享的挑战,从而改善了模型性能.
- 该研究为更强大,更有效的联合培训系统铺平了道路,能够适应各种数据分布和模型架构.
相关概念视频
Distillation: Vapor–Liquid Equilibria
2.8K
Distillation is a separation technique that takes advantage of the boiling point properties of disparate elements in a mixture. To perform distillation, we begin by heating a miscible mixture of two liquids with a significant difference in boiling points (at least 20°C). As the solution heats up and reaches the bubble point of the more volatile component, some molecules of the more volatile component transition into the gas phase and travel upward into the condenser, which is a glass tube...
2.8K
Social Proof
27.7K
Social proof is a form of persuasion based on comparison and conformity. People compare their behavior and actions to what others are doing and will change to conform to do what their peers do.
27.7K
Protecting Groups for Aldehydes and Ketones: Introduction
7.1K
Protecting groups are compounds that can bind to a specific functional group in the presence of other functional groups to protect them from undesired chemical reactions. These compounds can selectively bind to particular functional groups and advance chemoselective reactions in polyfunctional systems (Figure 1). After the functional group has served its purpose, it is removed by reacting it with specific compounds.
7.1K
Theories of Dissolution: Diffusion Layer Model
766
Dissolution, the process by which drug particles dissolve in a solvent, is explained by the diffusion layer model, a theoretical framework that simulates the absorption of oral drugs and allows us to analyze experimental data.
This process starts with a thin layer, saturated with the drug, forming at the interface between the solid and liquid. The solute then diffuses from this layer into the main solution. The Noyes-Whitney equation suggests that the rate of dissolution relies on the diffusion...
This process starts with a thin layer, saturated with the drug, forming at the interface between the solid and liquid. The solute then diffuses from this layer into the main solution. The Noyes-Whitney equation suggests that the rate of dissolution relies on the diffusion...
766
Social Facilitation
32.0K
Not all intergroup interactions lead to negative outcomes. Sometimes, being in a group situation can improve performance. Social facilitation occurs when an individual performs better when an audience is watching than when the individual performs the behavior alone. This typically occurs when people are performing a task for which they are skilled.
32.0K
¹³C NMR: Distortionless Enhancement by Polarization Transfer (DEPT)
1.1K
When proton-coupled carbon-13 spectra are simplified by a broadband proton decoupling technique, structural information about the coupled protons is lost. Distortionless enhancement by polarization transfer (DEPT) is a technique that provides information on the number of hydrogens attached to each carbon in a molecule. While the DEPT experiment utilizes complex pulse sequences, the pulse delay and flip angle are specifically manipulated. The resulting signals have different phases depending on...
1.1K


