FedTKD:基于适应性知识蒸的可靠的异质联合学习
Leiming Chen1, Weishan Zhang1, Cihao Dong1
1School of Computer Science and Technology, China University of Petroleum (East China), Qingdao 266580, China.
Entropy (Basel, Switzerland)
|January 26, 2024
概括
本研究介绍了FedTKD,这是一个可靠的,用于异质模型的联合学习框架. 它可以识别恶意客户端,并选择性地融合知识,在不同的环境中提高模型准确性和隐私.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 保护隐私的技术 保护隐私的技术
背景情况:
- 传统的联合学习 (FL) 需要同质的模型结构,限制其在现实世界异质环境中的应用.
- 现有的异质FL知识蒸方法通常假定客户可信度,未能解决恶意或低质量的数据贡献.
- 在FL中整合个性化模型是具有挑战性的,因为模型异质性和需要可靠的知识聚合.
研究的目的:
- 提出一个值得信赖的异质联合学习框架 (FedTKD),解决客户识别和可靠的知识融合问题.
- 在具有多样化的客户端模型结构和潜在恶意参与者的环境中实现联合学习.
- 在异质条件下提高联合模型的准确性和稳定性.
主要方法:
- 开发了一种恶意客户端识别方法,使用客户端logit功能来过不可靠的信息.
- 实施了选择性知识融合技术,用于高质量的全球逻辑计算.
- 引入了适应性知识蒸方法,以改善服务器对客户端的知识传输.
主要成果:
- 与基线方法相比,FedTKD在各种攻击和数据分布场景中表现出优越的性能.
- 该框架在不同的攻击策略下也表现出稳定的性能.
- 在具有异质数据分布的联合模型中实现了2%至3%的准确性改进.
结论:
- FedTKD有效地解决了在异构的联合学习环境中可靠的知识融合的挑战.
- 提出的客户身份识别和选择性知识聚合方法提高了模型可靠性和隐私.
- 该框架为与多样化且可能不值得信赖的客户的实用联合学习应用提供了强大的解决方案.
相关概念视频
Associative Learning
370
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
370
Distillation: Vapor–Liquid Equilibria
2.8K
Distillation is a separation technique that takes advantage of the boiling point properties of disparate elements in a mixture. To perform distillation, we begin by heating a miscible mixture of two liquids with a significant difference in boiling points (at least 20°C). As the solution heats up and reaches the bubble point of the more volatile component, some molecules of the more volatile component transition into the gas phase and travel upward into the condenser, which is a glass tube...
2.8K
Distribution Reliability and Automation
107
Distribution reliability in electrical power systems is critical for ensuring an uninterrupted power supply to consumers at minimal cost. According to IEEE Standard Terms, reliability is the probability that a device will function without failure over a specified time period or amount of usage. For electric power distribution, this translates to maintaining continuous power supply and addressing customer concerns over power outages. Several indices, as defined by IEEE Standard 1366-2012, are...
107
Generalization, Discrimination, and Extinction
559
Generalization, discrimination, and extinction are key concepts in operant conditioning that influence how behaviors are learned and maintained.
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
559
Improving Translational Accuracy
10.5K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
10.5K
Model Approaches for Pharmacokinetic Data: Distributed Parameter Models
69
Pharmacokinetic models are mathematical constructs that represent and predict the time course of drug concentrations in the body, providing meaningful pharmacokinetic parameters. These models are categorized into compartment, physiological, and distributed parameter models.
The distributed parameter models are specifically designed to account for variations and differences in some drug classes. This model is particularly useful for assessing regional concentrations of anticancer or...
The distributed parameter models are specifically designed to account for variations and differences in some drug classes. This model is particularly useful for assessing regional concentrations of anticancer or...
69


