联合学习与随机交替方向方法的乘数和在训练神经网络中的应用
Duy Nhat Phan1, Patrick Hytla1, Andrew Rice1
1University of Dayton Research Institute, University of Dayton, 300 College Park, Dayton, 45469, OH, USA.
概括
本研究介绍了一种新的联合学习 (FL) 算法,使用ADMM和最大化-最小化来实现高效的私人模式培训. 它实现了最佳的通信复杂性,并在神经网络分类中展示了实际的有效性.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 分布式计算 (Distributed Computing) 是一种分布式计算.
背景情况:
- 联合学习 (FL) 允许在多个用户之间进行分散的模型培训,同时确保数据隐私.
- 在FL的优化挑战,如非凸度和非光滑度,导致显著的计算困难.
- 现有的FL算法经常在融合和通信效率方面扎.
研究的目的:
- 提出一种新的联合学习算法,解决固有的优化挑战.
- 提高联合学习的融合特性和沟通效率.
- 在现实场景中证明拟议的算法在实际应用中的应用性.
主要方法:
- 拟议的算法将乘数交替方向方法 (ADMM) 与随机区块坐标策略集成在一起.
- 一般的最大化-最小化原理用于处理非凸和非光滑的优化问题.
- 理论分析提供了几乎确定的后续趋同到静止点的保证.
主要成果:
- 该算法实现了最知名的复杂性,在联合学习中限制了通信回合的数量.
- 对基准数据集的实证评估验证了算法的有效性.
- 该方法在使用神经网络的分类任务中表现出强的表现.
结论:
- 新的FL算法有效地解决了分散式学习中的优化挑战.
- 与现有方法相比,拟议的方法提供了更高的通信效率.
- 该算法在现实应用中显示出实际有效性,特别是在基于神经网络的分类中.
相关概念视频
Randomized Experiments
6.6K
The randomization process involves assigning study participants randomly to experimental or control groups based on their probability of being equally assigned. Randomization is meant to eliminate selection bias and balance known and unknown confounding factors so that the control group is similar to the treatment group as much as possible. A computer program and a random number generator can be used to assign participants to groups in a way that minimizes bias.
Simple randomization
Simple...
Simple randomization
Simple...
6.6K
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
29
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
29


