具有沟通效率的联合学习与阶段性培训战略.
Yifei Cheng1, Shuheng Shen2, Xianfeng Liang3
1Anhui Province Key Lab of Big Data Analysis and Application, China; School of Data Science, University of Science and Technology of China, China; State Key Laboratory of Cognitive Intelligence, China.
概括
本研究介绍了SQUARFA,这是一种提高联合学习通信效率的算法,特别是与非IID数据. 它实现了最佳的融合和通信复杂性,解决了分布式机器学习的一个关键挑战.
科学领域:
- 机器学习 机器学习
- 分布式系统 分布式系统
- 优化算法 优化算法
背景情况:
- 联合学习 (FL) 的表现取决于沟通效率.
- 非独立和相同分布的数据 (非IID) 显著增加了FL的通信成本.
- 现有的减少差异的方法难以同时实现最佳的通信复杂性和收率.
研究的目的:
- 开发一种优化算法,克服当前联合学习沟通策略的局限性.
- 为了实现 FL 与非 IID 数据同时实现最佳的通信复杂性和融合率.
- 为 FL.中的凸和非凸优化问题提供理论保证.
主要方法:
- 拟议的SQUARFA算法具有阶段性训练框架,具有减小差异和快速启动阶段.
- 开发了一种用于一般非凸的目标的SQUARFA变体.
- 将SQUARFA技术扩展到大批量设置.
主要成果:
- 在PL条件下,SQUARFA实现了强凸和非凸目标的最佳融合率和通信复杂性.
- 一种SQUARFA的变体为一般非凸的目标提供了最佳的理论结果.
- 大批量扩展的SQUARFA实现了最佳的通信复杂性.
结论:
- SQUARFA有效地解决了联合学习中的沟通效率挑战,特别是使用非IID数据.
- 拟议的算法同时提供最佳的融合和通信复杂性,填补了FL研究中的关键缺口.
- 实验验证证证实了SQUARFA及其变体的卓越性能.
相关概念视频
Survival Tree
105
Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
Building a Survival Tree
Constructing a...
Building a Survival Tree
Constructing a...
105
Improving Translational Accuracy
11.5K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
11.5K
Observational Learning
207
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
207
Associative Learning
434
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
434
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
79
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
79
Load-frequency control
187
Load-frequency control (LFC) is vital for maintaining power system stability, ensuring that frequency and power flows remain within acceptable limits during load changes. Turbine-governor control eliminates rotor accelerations and decelerations following load changes. However, a steady-state frequency error persists when the change in the turbine-governor reference setting is zero. In an interconnected power system, each area agrees to export or import a scheduled amount of power through...
187


