在分散的联合学习中,初始化和网络效应
Arash Badie-Modiri1,2, Chiara Boldrini3, Lorenzo Valerio3
1Department of Network and Data Science, Central European University, 1100 Vienna, Austria.
概括
完全去中心化的联合学习列车在本地数据上协作模式,增强隐私. 一个基于网络拓学的新初始化策略显著提高了人工神经网络的训练效率.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 网络科学 网络科学
背景情况:
- 分散的联合学习 (DFL) 允许在没有中央协调的情况下进行协作模式培训,从而保护数据隐私.
- DFL的性能对网络拓和初始模型条件敏感.
- 现有的DFL方法往往缺乏高效,无协调的初始化策略.
研究的目的:
- 为DFL提出一个新的初始化策略,利用网络拓.
- 提高分散的人工神经网络的培训效率和可扩展性.
- 调查网络结构对DFL动态的影响.
主要方法:
- 开发了基于自身向量中心分布的人工神经网络的无协调初始化策略.
- 分析了网络拓对DFL性能的影响.
- 在拟议的初始化下研究了缩放行为和参数选择.
主要成果:
- 拟议的初始化策略显著提高了去中心化联合学习的效率.
- 证明了网络拓,初始化和学习动态之间的明显联系.
- 为拟议的初始化战略确定了最佳的环境参数.
结论:
- 网络结构和初始化对于高效的DFL至关重要.
- 拟议的方法为分散的AI培训提供了可扩展和有效的方法.
- 这项研究为设计强大的DFL系统提供了基础的见解.
相关概念视频
Randomized Experiments
8.8K
The randomization process involves assigning study participants randomly to experimental or control groups based on their probability of being equally assigned. Randomization is meant to eliminate selection bias and balance known and unknown confounding factors so that the control group is similar to the treatment group as much as possible. A computer program and a random number generator can be used to assign participants to groups in a way that minimizes bias.
Simple randomization
Simple...
Simple randomization
Simple...
8.8K
Distributed Loads: Problem Solving
1.1K
Beams are structural elements commonly employed in engineering applications requiring different load-carrying capacities. The first step in analyzing a beam under a distributed load is to simplify the problem by dividing the load into smaller regions, which allows one to consider each region separately and calculate the magnitude of the equivalent resultant load acting on each portion of the beam. The magnitude of the equivalent resultant load for each region can be determined by calculating...
1.1K
Introduction to Learning
917
Learning is the process of acquiring knowledge or skills through practice or experience, leading to long-lasting behavioral changes. This acquisition occurs through interaction with the environment and requires practice or experience. For instance, mastering a skill such as surfing requires considerable practice and experience, highlighting the essential role of repeated interactions with the environment in learning.
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
917
Associative Learning
1.2K
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
1.2K
Cognitive Learning
991
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
991
Observational Learning
807
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
807


