更严格的遗憾在线联合学习的分析和优化
IEEE transactions on pattern analysis and machine intelligence
|September 19, 2023
概括
本研究介绍了OFedIQ,这是一种在线联合学习 (OFL) 的沟通效率高的方法. 它可以显著降低99%的通信成本,同时在分布式流数据场景中保持模型性能.
科学领域:
- 机器学习 机器学习
- 分布式系统 分布式系统
- 优化优化 优化优化
背景情况:
- 联合学习 (FL) 通常假定离线数据,但实际应用需要在线处理流数据.
- 在线FL (OFL) 通过从分布式数据流中学习序列模型来解决这个问题,以尽量减少累积遗憾.
- 在OFL中标准的FedOGD方法是通信密集的.
研究的目的:
- 开发一种沟通效率高的在线联合学习 (OFL) 方法.
- 导出一个遗憾的结合,解释数据异质性和通信效率技术.
- 优化参数以提高性能和减少通信开销.
主要方法:
- 推出了OFedIQ,一个通信效率高的OFL算法.
- 采用间歇性传输 (客户端子采样,周期性传输) 和梯度量化.
- 鉴于数据异质性和通信效率,我们得出了一个新的遗憾.
主要成果:
- OFedIQ实现了与FedOGD可比的异常表现.
- 证明了99%的通信成本降低.
- 在各种在线ML任务中对现实世界数据集的验证有效性.
结论:
- OFedIQ为在线联合学习提供了一种实用而高效的解决方案.
- 拟议的方法平衡了模型性能与显著的通信节省.
- 对于需要实时预测的分布式流数据场景有效.
相关概念视频
Improving Translational Accuracy
11.5K
Base complementarity between the three base pairs of mRNA codon and the tRNA anticodon is not a failsafe mechanism. Inaccuracies can range from a single mismatch to no correct base pairing at all. The free energy difference between the correct and nearly correct base pairs can be as small as 3 kcal/ mol. With complementarity being the only proofreading step, the estimated error frequency would be one wrong amino acid in every 100 amino acids incorporated. However, error frequencies observed in...
11.5K
Reducing Line Loss
173
In a three-phase circuit, line loss is an indicator of energy dissipated as heat due to the resistance of transmission lines. To address this, incorporating transformers into the system—a step-up transformer at the source and a step-down transformer at the load—is a strategic solution. Two three-phase transformers are introduced to improve this.
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss...
With a step-up transformer at the source, the voltage is increased, thereby reducing the current in the transmission lines since power loss...
173
Prediction Intervals
2.3K
The interval estimate of any variable is known as the prediction interval. It helps decide if a point estimate is dependable.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y.
2.3K
Observational Learning
207
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
207
Propagation of Uncertainty from Random Error
724
An experiment often consists of more than a single step. In this case, measurements at each step give rise to uncertainty. Because the measurements occur in successive steps, the uncertainty in one step necessarily contributes to that in the subsequent step. As we perform statistical analysis on these types of experiments, we must learn to account for the propagation of uncertainty from one step to the next. The propagation of uncertainty depends on the type of arithmetic operation performed on...
724
Regression Toward the Mean
6.3K
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when...
6.3K


