联合学习与贝叶斯神经网络相遇:强大的和不确定性意识分布式变量推理
Pengfei Li1, Qinghua Hu2, Xiaofei Wang2
1College of Intelligence and Computing, Tianjin University, Tianjin, 300350, China; School of Intelligence Science and Engineering, Qinghai Minzu University, Xining, 810007, China.
概括
联合学习 (FL) 与不确定性意识贝叶斯神经网络 (BNNs) 提高了模型稳定性和数据隐私. 这种新的FedUAB方法解决了客户端数据的限制和异质性,以实现分布式机器学习的卓越性能.
科学领域:
- 机器学习 机器学习
- 人工智能的人工智能
- 数据 隐私 数据 隐私 数据
背景情况:
- 联合学习 (FL) 广泛用于数据隐私,但与有限的客户端数据和异质性作斗争,导致低于最佳的聚合模型.
- 客户端模型过拟合和漂移是传统FL框架中的重大挑战.
- 现有的FL方法往往无法充分解决分布式,异质数据集的复杂性.
研究的目的:
- 引入一种新的联合学习方法,集成贝叶斯神经网络 (BNNs),以增强模型的稳定性和数据隐私.
- 解决将BNN与FL合并的关键挑战,包括事先选择,重量聚合和差异管理.
- 在模拟FL环境中提高全球和个性化模型的性能.
主要方法:
- 开发了FedUAB (具有不确定性意识的BNN的FL),其中客户使用贝叶斯的背向螺旋算法训练BNN.
- 模拟的BNN权重作为高斯分布,以减轻过度匹配和增强数据隐私.
- 实施了先前分布选择,高斯重量聚合和差异管理的新方法.
主要成果:
- 在模拟中,FedUAB在传统的FL和其他贝叶斯式FL方法中表现出优越的性能.
- 这种方法提高了稳定性,并减轻了FL固有的过拟合问题.
- FedUAB模型有效量化和分解不确定性,提供了有价值的见解.
结论:
- 通过集成不确定性意识的BNN,FedUAB方法为保护隐私的分布式机器学习提供了强大的解决方案.
- 这种方法有效地解决了FL的数据限制和异质性挑战.
- 对于FL系统来说,FedUAB提供了增强的性能和不确定性量化能力.
相关概念视频
00:59Propagation of Uncertainty from Random Error
635
An experiment often consists of more than a single step. In this case, measurements at each step give rise to uncertainty. Because the measurements occur in successive steps, the uncertainty in one step necessarily contributes to that in the subsequent step. As we perform statistical analysis on these types of experiments, we must learn to account for the propagation of uncertainty from one step to the next. The propagation of uncertainty depends on the type of arithmetic operation performed on...
635
01:10Propagation of Uncertainty from Systematic Error
462
The atomic mass of an element varies due to the relative ratio of its isotopes. A sample's relative proportion of oxygen isotopes influences its average atomic mass. For instance, if we were to measure the atomic mass of oxygen from a sample, the mass would be a weighted average of the isotopic masses of oxygen in that sample. Since a single sample is not likely to perfectly reflect the true atomic mass of oxygen for all the molecules of oxygen on Earth, the mass we obtain from this...
462
00:59Uncertainty: Overview
508
In analytical chemistry, we often perform repetitive measurements to detect and minimize inaccuracies caused by both determinate and indeterminate errors. Despite the cares we take, the presence of random errors means that repeated measurements almost never have exactly the same magnitude. The collective difference between these measurements - observed values - and the estimated or expected value is called uncertainty. Uncertainty is conventionally written after the estimated or expected value.
508
01:37Neural Regulation
39.1K
Digestion begins with a cephalic phase that prepares the digestive system to receive food. When our brain processes visual or olfactory information about food, it triggers impulses in the cranial nerves innervating the salivary glands and stomach to prepare for food.
39.1K
00:54Uncertainty: Confidence Intervals
3.1K
The confidence interval is the range of values around the mean that contains the true mean. It is expressed as a probability percentage. The interpretation of a 95% confidence interval, for instance, is that the statistician is 95% confident that the true mean falls within the interval. The upper and lower limits of this range are known as confidence limits. The confidence limits for the true mean are estimated from the sample's mean, the standard deviation, and the statistical factor...
3.1K
01:25Distribution Reliability and Automation
105
Distribution reliability in electrical power systems is critical for ensuring an uninterrupted power supply to consumers at minimal cost. According to IEEE Standard Terms, reliability is the probability that a device will function without failure over a specified time period or amount of usage. For electric power distribution, this translates to maintaining continuous power supply and addressing customer concerns over power outages. Several indices, as defined by IEEE Standard 1366-2012, are...
105