稳定和加速对异质数据的联合学习,部分客户参与
IEEE transactions on pattern analysis and machine intelligence
|September 26, 2024
概括
联邦学习 (FL) 稳定性通过测量客户端模型差异的新指数得到改善. 费达纳格算法增强了稳定性,加快了趋同,即使数据异质.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 分布式系统 分布式系统
背景情况:
- 联合学习 (FL) 允许在不集中数据的情况下进行协作模式培训.
- 每个客户端的多个本地更新减少了通信开销,但可能会破坏全球融合的稳定.
- 客户之间的数据异质性是FL的不稳定的主要驱动因素.
研究的目的:
- 为联合学习引入一种新的本地更新稳定性指数.
- 分析本地更新对FL稳定性和趋同性的影响.
- 提出一个加速和稳定的FL算法.
主要方法:
- 定义了基于客户端间模型差异的本地更新稳定性指数.
- 理论上分析了最先进的FL方法的稳定性.
- 开发了FedANAG,一种使用服务器和客户端级别Nesterov加速梯度 (NAG) 的FL算法.
主要成果:
- 局部更新稳定性指数量化了客户端模型变化的全球模型的影响.
- 通过使用全球势头,FedANAG提高了本地更新稳定性.
- 在各种数据异质性和参与率上,FedANAG显示了加速的融合和更高的准确性.
结论:
- 在FL的本地更新可能会对稳定性和融合产生负面影响,特别是数据异质性.
- 拟议的稳定性指数为FL方法的局限性提供了见解.
- 费达纳格为稳定高效的联合学习提供了一个有前途的解决方案.
相关概念视频
Associative Learning
309
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
309
Parallel Processing
145
The brain processes sensory information rapidly due to parallel processing, which involves sending data across multiple neural pathways at the same time. This method allows the brain to manage various sensory qualities, such as shapes, colors, movements, and locations, all concurrently. For instance, when observing a forest landscape, the brain simultaneously processes the movement of leaves, the shapes of trees, the depth between them, and the various shades of green. This enables a quick and...
145
Multicompartment Models: Overview
112
Multicompartment models are mathematical constructs that depict how drugs are distributed and eliminated within the body. They segment the body into several compartments, symbolizing various physiological or anatomical areas connected through drug transfer processes such as absorption, metabolism, distribution, and elimination.
These models offer a more comprehensive representation of drug behavior in the body than one-compartment models. They accommodate the complexity of drug distribution,...
These models offer a more comprehensive representation of drug behavior in the body than one-compartment models. They accommodate the complexity of drug distribution,...
112
Improving Translational Accuracy
9.4K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
9.4K
Model Approaches for Pharmacokinetic Data: Distributed Parameter Models
64
Pharmacokinetic models are mathematical constructs that represent and predict the time course of drug concentrations in the body, providing meaningful pharmacokinetic parameters. These models are categorized into compartment, physiological, and distributed parameter models.
The distributed parameter models are specifically designed to account for variations and differences in some drug classes. This model is particularly useful for assessing regional concentrations of anticancer or...
The distributed parameter models are specifically designed to account for variations and differences in some drug classes. This model is particularly useful for assessing regional concentrations of anticancer or...
64
Distribution Reliability and Automation
107
Distribution reliability in electrical power systems is critical for ensuring an uninterrupted power supply to consumers at minimal cost. According to IEEE Standard Terms, reliability is the probability that a device will function without failure over a specified time period or amount of usage. For electric power distribution, this translates to maintaining continuous power supply and addressing customer concerns over power outages. Several indices, as defined by IEEE Standard 1366-2012, are...
107


