DFedGFM:通过全球平面性和全球势头,追求分散式联合学习的全球一致性
Qinglun Li1, Miao Zhang1, Tao Sun1
1National University of Defense Technology, No. 109 Deya Road, Chang Sha, 410073, Hu Nan, China.
概括
像DFedGFM这样的分散式联合学习 (DFL) 算法通过使用全球信息来解决客户端漂移,以获得更好的一致性. 这种新的方法提高了对异质数据的稳定性,并实现了更紧密的融合率.
科学领域:
- 机器学习 机器学习
- 分布式系统 分布式系统
背景情况:
- 集中式联合学习 (CFL) 面临通信成本和隐私问题.
- 分散联合学习 (DFL) 提供了一个替代方案,但由于数据异质性和不一致性而遭受客户端漂移.
- 现有的DFL方法主要侧重于本地更新,忽视了减轻客户漂移至关重要的全球信息.
研究的目的:
- 提出一个新的DFL算法,DFedGFM,同时探索全球平度和动量.
- 提高DFL的整体一致性和稳定性,特别是在异质数据方面.
- 从理论和实验上验证DFedGFM与现有DFL算法的有效性.
主要方法:
- 在现有的DFL优化器中重新考虑局部平度和动量加速.
- 开发DFedGFM以利用全球梯度估计来识别全球平面.
- 整合全球势头来提高对异质数据的稳定性.
主要成果:
- 与DFedSAM相比,DFedGFM显示了与DFedSAM相比更好的全球平面性识别.
- 由于全球势头,该算法对异质数据具有增强的稳定性,优于DFedAvgM.
- 理论分析证明了DFedGFM的O1T融合率,随着通信拓连接的改善,边界越来越紧张.
结论:
- 通过利用全球信息,DFedGFM有效地解决了DFL的客户漂移问题.
- 与现有的DFL方法相比,拟议的算法实现了更高的性能和融合率.
- DFedGFM为保护隐私和高效的去中心化机器学习提供了一个有前途的解决方案.
相关概念视频
Conservation of Momentum: Introduction
14.5K
The total momentum of a system consisting of N interacting objects is constant in time or is conserved. A system must meet two requirements for its momentum to be conserved:
14.5K
Linear Momentum
13.8K
The term momentum is used in various ways in everyday language, most of which are consistent with the precise scientific definition. Generally, momentum implies a tendency to continue on course—to move in the same direction; we tend to speak of sports teams or politicians gaining and maintaining the momentum to win. Momentum is also associated with great mass and speed and is often considered when talking about collisions. For example, when rugby players collide and fall to the...
13.8K
Linear Momentum in Control Volume
719
Newton's second law is applied to obtain the linear momentum in a control volume in a fluid system. According to this law, the rate of change of linear momentum is equal to the sum of external forces acting on the system. When a control volume matches the fluid system at a specific moment, the forces acting on both are identical. Reynolds transport theorem helps explain this by breaking down the system's linear momentum into two components: the rate of change of linear momentum within...
719
Principle of Moments
1.6K
The principle of moments, also known as Varignon's theorem, is a fundamental concept in physics and engineering that describes the equilibrium of a rigid body under the influence of external forces. The principle states that the moment of a force about a point is equal to the sum of the moments of the components of the force about the same point.
The moment is calculated by multiplying the magnitude of the force by the perpendicular distance from the point of application to the point about...
The moment is calculated by multiplying the magnitude of the force by the perpendicular distance from the point of application to the point about...
1.6K
Conservation of Linear Momentum for a System of Particles
214
In the dynamic realm of billiards, a fascinating interplay of forces governs the motion of cue balls and stationary balls. When the cue ball collides with a stationary ball, linear momentum is exchanged. The cue ball imparts a fraction of its linear momentum to the stationary ball, causing the cue ball to decelerate while initiating the motion of the stationary ball.
The impulsive force at play during this interaction is of extremely short duration, rendering its impulse negligible. When...
The impulsive force at play during this interaction is of extremely short duration, rendering its impulse negligible. When...
214
Resultant of a General Distributed Loading
641
While designing structures exposed to non-uniform loads, it is crucial to consider the resultant force and its location. This resultant force is a single vector representing the net force applied due to the distributed load.
Examples such as load distribution due to wind and load distribution on a bridge illustrate how this concept is used to analyze and design safe, reliable structures under variable loading conditions. Most structures, such as residential buildings, bridges, and towers, are...
Examples such as load distribution due to wind and load distribution on a bridge illustrate how this concept is used to analyze and design safe, reliable structures under variable loading conditions. Most structures, such as residential buildings, bridges, and towers, are...
641


