通过局部适应性修改优化器进行高效的联合学习,具有线性加速度.
概括
联邦局部ADaptive修改优化器 (FedLADA) 通过使用动量来纠正局部模型漂移并提高训练速度来增强联合学习. 这种新的方法实现了更高的准确性,并减少了分布式机器学习中的通信回合.
科学领域:
- 机器学习 机器学习
- 分布式系统 分布式系统
- 优化算法 优化算法
背景情况:
- 适应性优化在分布式学习中是成功的,但在扩展到联合学习 (FL) 时面临效率低下.
- 在FL的挑战包括来自不准确的全球梯度和客户端漂移由于局部过度装配的崎的融合.
- 现有的自适应优化器在FL中与分散的,异质的数据环境的独特需求作斗争.
研究的目的:
- 提出一个新的基于动量的算法,FedLADA,以解决联合学习优化的低效性.
- 提高实证培训速度,减轻联合学习环境中的异质过度适应.
- 从理论上确定拟议的算法的收率和线性加速度属性.
主要方法:
- 开发了联邦局部ADaptive修改优化器 (FedLADA),这是一个基于势头的算法,将全球梯度下降与局部修改的自适应优化器相结合.
- 根据FedLADA的估计和纠正,使用基于前一轮全球平均偏移的势头类术语来估计和纠正本地模式偏移.
- 理论分析确定了部分客户参与的非凸起案件的收率.
主要成果:
- 费德拉达 (FedLADA) 证明了经验训练速度的提高,并有效地减轻了异构的过度装配.
- 在现实数据集上的实验结果表明,FedLADA与基线相比,显著减少了通信轮次.
- 拟议的算法比现有的几个联合学习优化方法获得了更高的准确性.
结论:
- 在联邦式学习中,FedLADA为融合和客户端漂移问题提供了有效的解决方案.
- 该算法显示了加速培训和提高分散式学习环境中的模型准确性的承诺.
- FedLADA提供了一个理论上合理的和经验验证的方法,用于高效的联合学习优化.
更多相关视频
07:34A Simple Stimulatory Device for Evoking Point-like Tactile Stimuli: A Searchlight for LFP to Spike Transitions
Published on: March 25, 2014
9.9K
06:45Design and Application of a Fault Detection Method Based on Adaptive Filters and Rotational Speed Estimation for an Electro-Hydrostatic Actuator
Published on: October 28, 2022
1.7K
相关概念视频
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
81
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
81
Improving Translational Accuracy
11.6K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
11.6K
Load-frequency control
193
Load-frequency control (LFC) is vital for maintaining power system stability, ensuring that frequency and power flows remain within acceptable limits during load changes. Turbine-governor control eliminates rotor accelerations and decelerations following load changes. However, a steady-state frequency error persists when the change in the turbine-governor reference setting is zero. In an interconnected power system, each area agrees to export or import a scheduled amount of power through...
193
Linear Approximation in Frequency Domain
111
Linear systems are characterized by two main properties: superposition and homogeneity. Superposition allows the response to multiple inputs to be the sum of the responses to each individual input. Homogeneity ensures that scaling an input by a scalar results in the response being scaled by the same scalar.
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
In contrast, nonlinear systems do not inherently possess these properties. However, for small deviations around an operating point, a nonlinear system can often be approximated as linear....
111
Linear Approximation in Time Domain
101
Nonlinear systems often require sophisticated approaches for accurate modeling and analysis, with state-space representation being particularly effective. This method is especially useful for systems where variables and parameters vary with time or operating conditions, such as in a simple pendulum or a translational mechanical system with nonlinear springs.
For a simple pendulum with a mass evenly distributed along its length and the center of mass located at half the pendulum's length,...
For a simple pendulum with a mass evenly distributed along its length and the center of mass located at half the pendulum's length,...
101
Distributed Loads: Problem Solving
671
Beams are structural elements commonly employed in engineering applications requiring different load-carrying capacities. The first step in analyzing a beam under a distributed load is to simplify the problem by dividing the load into smaller regions, which allows one to consider each region separately and calculate the magnitude of the equivalent resultant load acting on each portion of the beam. The magnitude of the equivalent resultant load for each region can be determined by calculating...
671
