灵活的垂直联合学习与异质的党派
IEEE transactions on neural networks and learning systems
|September 22, 2023
概括
灵活的垂直联合学习 (Flex-VFL) 能够在不同的分区数据集上进行协作模型培训. 这种新的算法适应不断变化的系统动态,改善分布式机器学习的融合.
科学领域:
- 分布式机器学习 分布式机器学习
- 联合学习算法联合学习算法
背景情况:
- 垂直联合学习 (VFL) 在数据上训练模型,数据在多个方之间垂直分割.
- 现有的VFL方法与异质系统作斗争,其中党特征 (速度,架构,优化器) 随着时间的推移而变化.
研究的目的:
- 引入灵活的垂直联合学习 (Flex-VFL),这是一种用于训练光滑,非凸函数的新型分布式算法.
- 为应对数据垂直分区的分布式系统中异质性和动态变化的挑战.
主要方法:
- 柔性-VFL采用平行块坐标下降 (P-BCD),其中每一方使用随机坐标下降训练一个模型分区.
- 提供了理论收分析,证明收率取决于群体速度和局部优化器参数.
主要成果:
- 从理论上来说,Flex-VFL的收率被群体速度和局部优化器参数所限制.
- 开发了Flex-VFL的自适应扩展,以根据变化的速度和优化器参数动态调整学习速度.
结论:
- 在异质环境中,Flex-VFL为分布式机器学习提供了有效的解决方案.
- Flex-VFL的自适应扩展增强了其在动态系统中的稳定性和性能,在融合时间中超过同步和异步VFL算法.
相关概念视频
Associative Learning
428
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
428
Multi-input and Multi-variable systems
122
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
122
Introduction to Learning
460
Learning is the process of acquiring knowledge or skills through practice or experience, leading to long-lasting behavioral changes. This acquisition occurs through interaction with the environment and requires practice or experience. For instance, mastering a skill such as surfing requires considerable practice and experience, highlighting the essential role of repeated interactions with the environment in learning.
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
460
Aggregates Classification
340
Aggregate classification is generally based on its size, petrographic characteristics, weight, and source. Size classification ranges from coarse to fine aggregates, defined by the size of the particles. Coarse aggregates are particles that do not pass through ASTM sieve No. 4, and aggregates that pass through the sieve are fine aggregates.
Petrographic classification groups aggregates based on common mineralogical characteristics. Some of the common mineral groups found in aggregates are...
Petrographic classification groups aggregates based on common mineralogical characteristics. Some of the common mineral groups found in aggregates are...
340
Observational Learning
202
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
202
Functional Classification of Joints
4.2K
Functional Classification of Joints
The functional classification of joints is determined by the amount of mobility between the adjacent bones. Joints are functionally classified as a synarthrosis or immobile joint, an amphiarthrosis or slightly moveable joint, or as a diarthrosis, a freely moveable joint. Fibrous and cartilaginous joints can be functionally classified as either synarthroses or amphiarthroses, whereas all synovial joints are classified as diarthroses.
Synarthrosis
An...
The functional classification of joints is determined by the amount of mobility between the adjacent bones. Joints are functionally classified as a synarthrosis or immobile joint, an amphiarthrosis or slightly moveable joint, or as a diarthrosis, a freely moveable joint. Fibrous and cartilaginous joints can be functionally classified as either synarthroses or amphiarthroses, whereas all synovial joints are classified as diarthroses.
Synarthrosis
An...
4.2K


