在模型-异质联合学习中改善概括和个性化
IEEE transactions on neural networks and learning systems
|August 12, 2024
概括
联合学习 (FL) 现在可以与FedTED平衡全球模型通用化和客户个性化. 这种新的方法解决了异质模型,大大改善了这两个方面.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 分布式系统 分布式系统
背景情况:
- 传统的联合学习 (FL) 假设同质的客户端模型,需要参数共享,这构成安全风险,忽视了个性化.
- 现有的FL方法难以平衡服务器模型概括与个人客户端个性化,特别是异质模型.
研究的目的:
- 为了应对挑战,以 heterogeneous 客户端模型在联合学习中确保概括和个性化.
- 引入一个新的联合学习框架,FedTED,能够处理不同的客户端模型和目标.
主要方法:
- FedTED使用双分支结构来管理异质模型.
- 无数据知识蒸 (DFKD) 是用来促进知识转移而没有原始数据的方法.
- 该框架协调来自异质客户的更新,以建立一个强大的全球模型.
主要成果:
- FedTED显著提高了个性化和通用化性能.
- 在概括方面实现了19.37%的改进,在个性化方面提高了高达9.76%.
- 在异构的FL场景中,在代表性算法上表现出优异的性能.
结论:
- 在异质环境中,FedTED有效地解决了传统FL的局限性.
- 拟议的框架成功地平衡了全球通用化和本地客户个性化的竞争目标.
- FedTED为更有效,更安全的联合学习应用提供了一个有前途的解决方案.
更多相关视频
08:05Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques
Published on: June 30, 2020
7.5K
12:55Multimodal Protocol for Assessing Metacognition and Self-Regulation in Adults with Learning Difficulties
Published on: September 27, 2020
8.4K
相关概念视频
Generalization, Discrimination, and Extinction
501
Generalization, discrimination, and extinction are key concepts in operant conditioning that influence how behaviors are learned and maintained.
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
501
Improving Translational Accuracy
9.6K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
9.6K
Associative Learning
324
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
324
Stereotype Content Model
14.0K
The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence...
14.0K
Modeling and Similitude
258
Scaled modeling is a fundamental technique in engineering, enabling the study of large and complex systems by creating smaller, manageable replicas that recreate critical characteristics of the original. In hydrology and civil infrastructure, for example, scaled models of dams help analyze water flow, turbulence, and pressure. This method allows for accurate predictions of real-world behavior within a controlled environment, significantly reducing the cost and time involved in full-scale...
258
Multicompartment Models: Overview
118
Multicompartment models are mathematical constructs that depict how drugs are distributed and eliminated within the body. They segment the body into several compartments, symbolizing various physiological or anatomical areas connected through drug transfer processes such as absorption, metabolism, distribution, and elimination.
These models offer a more comprehensive representation of drug behavior in the body than one-compartment models. They accommodate the complexity of drug distribution,...
These models offer a more comprehensive representation of drug behavior in the body than one-compartment models. They accommodate the complexity of drug distribution,...
118
