FedADMM-InSa:用于联合学习的不准确和自我适应的ADMM
Yongcun Song1, Ziqi Wang2, Enrique Zuazua3
1Department of Mathematics, City University of Hong Kong, Kowloon, Hong Kong, China.
概括
本研究介绍了FedADMM-InSa,这是一个改进的联合学习 (FL) 算法,可以提高模型准确性并降低计算负载. 它解决了FL中的超参数调整挑战,使分布式学习更有效和更强大.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 分布式系统 分布式系统
背景情况:
- 联合学习 (FL) 实现了维护隐私的分布式模型培训.
- 现有的FL方法,包括FedADMM,面临着异质数据/系统和超参数灵敏度的挑战.
- 有效的FL需要解决通信瓶和当地资源限制.
研究的目的:
- 开发一种更强大,更有效的联合学习算法,尽量减少超参数调整.
- 为了降低计算成本并减轻联合学习中的拖延效应.
- 提高跨多种数据集和系统的联合学习的整体性能和适用性.
主要方法:
- 提出FedADMM-InSa,一个不准确和自我适应的联合ADMM算法.
- 实施局部客户端更新的不准确性标准,独立于实证局部培训准确性.
- 引入一个自适应方案,以动态调整客户处罚参数.
主要成果:
- 与基准算法相比,FedADMM-InSa的模型准确度提高了7.8%.
- 客户端本地工作负载通过不准确性标准和适应性惩罚减少了55.7%.
- 该算法显示了对数据和系统异质性的弹性,而不需要广泛的超参数调整.
结论:
- FedADMM-InSa为联合学习提供了更有效和更强大的解决方案,特别是在异质的环境中.
- 拟议的不准确性标准和自适应性惩罚参数调整大大降低了计算开销,提高了性能.
- 这项工作通过简化超参数管理和增强实际应用性来推进联合学习.
相关概念视频
Associative Learning
303
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
303
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
44
Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
44
Accuracy, limits, and approximation
441
Accuracy, limits, and approximations are common in many fields, especially in engineering calculations. These concepts are imperative for ensuring that a given value is as close as possible to its true value.
Accuracy is defined as the closeness of the measured value to the true or actual value. In engineering mechanics, repeated measurements are taken during theoretical or experimental analyses to ensure that the result is precise and accurate.
The accuracy of any solution is based on the...
Accuracy is defined as the closeness of the measured value to the true or actual value. In engineering mechanics, repeated measurements are taken during theoretical or experimental analyses to ensure that the result is precise and accurate.
The accuracy of any solution is based on the...
441
Improving Translational Accuracy
9.3K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
9.3K
Mean Absolute Deviation
2.6K
The mean absolute deviation is also a measure of the variability of data in a sample. It is the absolute value of the average difference between the data values and the mean.
Let us consider a dataset containing the number of unsold cupcakes in five shops: 10, 15, 8, 7, and 10. Initially, calculate the sample mean. Then calculate the deviation, or the difference, between each data value and the mean. Next, the absolute values of these deviations are added and divided by the sample size to...
Let us consider a dataset containing the number of unsold cupcakes in five shops: 10, 15, 8, 7, and 10. Initially, calculate the sample mean. Then calculate the deviation, or the difference, between each data value and the mean. Next, the absolute values of these deviations are added and divided by the sample size to...
2.6K
Multi-input and Multi-variable systems
100
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
100


