具有异质性意识的高效联合学习与混合同步-异步分割策略
Zijian Li1, Boyuan Li2, Kunyu Zhang2
1College of Artificial Intelligence, Dalian Maritime University, Dalian, China.
概括
联合学习 (FL) 面临来自设备异质性的挑战. 我们的HA-HEFL框架通过为各种设备定制模型来平衡效率和准确性,从而提高培训成果.
科学领域:
- 人工智能
- 机器学习
- 分布式系统
背景情况:
- 联合学习 (FL) 能够在保护数据隐私的同时提供协作模式的培训.
- 在FL的系统异质性导致"滞后者问题",由于资源有限的设备延迟了聚合.
- 目前的解决方案往往忽视边缘约束,有可能导致模型偏差和数据遗漏.
研究的目的:
- 推出HA-HEFL,一个具有异质性意识的高效联合学习框架.
- 解决FL的培训效率,模型准确性和资源消耗之间的权衡问题.
- 减轻系统异质性对FL性能的负面影响.
主要方法:
- 使用神经元级别的配置和基于优先级的选择来实现资源意识的自适应模型定制.
- 混合同步-异步分割培训与知识蒸用于特征提取和分类器更新.
- 根据基线优先加权汇总,以确保均衡的全球模型更新.
主要成果:
- 与现有方法相比,HA-HEFL显著提高了融合速度.
- 该框架提高了模型的整体准确性.
- HA-HEFL显示了网络流量的显著减少.
结论:
- 在异质的FL环境中,HA-HEFL有效地平衡了培训效率,模型准确性和资源消耗.
- 拟议的自适应型定制和混合培训策略克服了落后问题.
- HA-HEFL为各种边缘设备的实际FL部署提供了一种优越的方法.
相关概念视频
Stratified Sampling Method
12.8K
Sampling is a technique to select a portion (or subset) of the larger population and study that portion (the sample) to gain information about the population. The sampling method ensures that samples are drawn without bias and accurately represent the population. Because measuring the entire population in a study is not practical, researchers use samples to represent the population of interest.
To choose a stratified sample, divide the population into groups called strata and then take a...
To choose a stratified sample, divide the population into groups called strata and then take a...
12.8K
Cluster Sampling Method
12.7K
Appropriate sampling methods ensure that samples are drawn without bias and accurately represent the population. Because measuring the entire population in a study is not practical, researchers use samples to represent the population of interest.
To choose a cluster sample, divide the population into clusters (groups) and then randomly select some of the clusters. All the members from these clusters are in the cluster sample. For example, if you randomly sample four departments from your...
To choose a cluster sample, divide the population into clusters (groups) and then randomly select some of the clusters. All the members from these clusters are in the cluster sample. For example, if you randomly sample four departments from your...
12.7K
Distributed Loads: Problem Solving
729
Beams are structural elements commonly employed in engineering applications requiring different load-carrying capacities. The first step in analyzing a beam under a distributed load is to simplify the problem by dividing the load into smaller regions, which allows one to consider each region separately and calculate the magnitude of the equivalent resultant load acting on each portion of the beam. The magnitude of the equivalent resultant load for each region can be determined by calculating...
729
Parallel Processing
224
The brain processes sensory information rapidly due to parallel processing, which involves sending data across multiple neural pathways at the same time. This method allows the brain to manage various sensory qualities, such as shapes, colors, movements, and locations, all concurrently. For instance, when observing a forest landscape, the brain simultaneously processes the movement of leaves, the shapes of trees, the depth between them, and the various shades of green. This enables a quick and...
224
Distribution Reliability and Automation
153
Distribution reliability in electrical power systems is critical for ensuring an uninterrupted power supply to consumers at minimal cost. According to IEEE Standard Terms, reliability is the probability that a device will function without failure over a specified time period or amount of usage. For electric power distribution, this translates to maintaining continuous power supply and addressing customer concerns over power outages. Several indices, as defined by IEEE Standard 1366-2012, are...
153
Associative Learning
569
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
569
