ヘテロゲニティ意識の高効率フェデレーション式学習とハイブリッドシンクロン・アシンクロン分割戦略
Zijian Li1, Boyuan Li2, Kunyu Zhang2
1College of Artificial Intelligence, Dalian Maritime University, Dalian, China.
まとめ
統合学習 (FL) は,デバイスの異質性による課題に直面しています. 私たちのHA-HEFLフレームワークは 効率と精度をバランスにし 様々なデバイスのモデルをカスタマイズして 訓練結果を改善します
科学分野:
- 人工知能
- 機械学習
- 分散型システム
背景:
- フェデラート・ラーニング (FL) は,データプライバシーを保ちながら,コラボレーションモデルトレーニングを可能にします.
- FLにおけるシステムの異質性は,資源の限られたデバイスによる集積を遅らせる"滞留者問題"につながります.
- 現在のソリューションでは,エッジの制約が無視され,モデルバイアスやデータ省略の危険性があります.
研究 の 目的:
- ハーテロゲニティに意識した高効率の統合学習の枠組みであるHA-HEFLを導入する.
- FLにおける訓練効率,モデル精度,資源消費の間のトレードオフに対処する.
- FL性能に対するシステムの異質性の負の影響を軽減する.
主な方法:
- ニューロンレベルのプロファイリングと優先度に基づく選択を使用して,リソースに配慮した適応モデルカスタマイズ.
- ハイブリッドシンクロン・アシンクロン・スプリット・トレーニングで,特性の抽出と分類器の更新のための知識の蒸留.
- バランスのとれたグローバルモデルの更新を確実にするため,ベースラインで優先順位付けされた加算.
主要な成果:
- HA-HEFLは,既存の方法と比較して,収束速度を大幅に改善します.
- フレームワークはモデルの全体的な精度を高めます.
- HA-HEFLは,ネットワークのトラフィックを大幅に減少させています.
結論:
- HA-HEFLは,異質なFL環境でトレーニングの効率,モデルの精度,リソースの消費を効果的にバランスします.
- 提案された適応モデルカスタマイズとハイブリッドトレーニング戦略は後退者の問題を克服します.
- HA-HEFLは,様々なエッジデバイスを使用した実用的なFL展開に優れたアプローチを提供します.
関連する概念動画
01:16Stratified Sampling Method
12.8K
Sampling is a technique to select a portion (or subset) of the larger population and study that portion (the sample) to gain information about the population. The sampling method ensures that samples are drawn without bias and accurately represent the population. Because measuring the entire population in a study is not practical, researchers use samples to represent the population of interest.
To choose a stratified sample, divide the population into groups called strata and then take a...
To choose a stratified sample, divide the population into groups called strata and then take a...
12.8K
01:20Cluster Sampling Method
12.7K
Appropriate sampling methods ensure that samples are drawn without bias and accurately represent the population. Because measuring the entire population in a study is not practical, researchers use samples to represent the population of interest.
To choose a cluster sample, divide the population into clusters (groups) and then randomly select some of the clusters. All the members from these clusters are in the cluster sample. For example, if you randomly sample four departments from your...
To choose a cluster sample, divide the population into clusters (groups) and then randomly select some of the clusters. All the members from these clusters are in the cluster sample. For example, if you randomly sample four departments from your...
12.7K
01:21Distributed Loads: Problem Solving
729
Beams are structural elements commonly employed in engineering applications requiring different load-carrying capacities. The first step in analyzing a beam under a distributed load is to simplify the problem by dividing the load into smaller regions, which allows one to consider each region separately and calculate the magnitude of the equivalent resultant load acting on each portion of the beam. The magnitude of the equivalent resultant load for each region can be determined by calculating...
729
01:20Parallel Processing
224
The brain processes sensory information rapidly due to parallel processing, which involves sending data across multiple neural pathways at the same time. This method allows the brain to manage various sensory qualities, such as shapes, colors, movements, and locations, all concurrently. For instance, when observing a forest landscape, the brain simultaneously processes the movement of leaves, the shapes of trees, the depth between them, and the various shades of green. This enables a quick and...
224
01:25Distribution Reliability and Automation
153
Distribution reliability in electrical power systems is critical for ensuring an uninterrupted power supply to consumers at minimal cost. According to IEEE Standard Terms, reliability is the probability that a device will function without failure over a specified time period or amount of usage. For electric power distribution, this translates to maintaining continuous power supply and addressing customer concerns over power outages. Several indices, as defined by IEEE Standard 1366-2012, are...
153
01:27Associative Learning
569
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
569