DSFedCon:用于数据驱动智能系统的动态稀疏联合对比学习
IEEE transactions on neural networks and learning systems
|January 26, 2024
概括
联合学习 (FL) 通过在不共享原始数据的情况下协作训练模型来增强数据隐私. 我们新的动态稀疏联合对比学习 (DSFedCon) 框架提高了准确性,并大大降低了非IID数据的通信成本.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 数据 隐私 数据 隐私 数据
背景情况:
- 联合学习 (FL) 能够在多个客户端进行协作模型培训,通过传达模型而不是原始数据来增强数据安全性.
- 然而,FL面临着一些挑战,包括对非独立且相同分布的 (非IID) 数据的准确性低,以及高计算/通信开销.
- 现有的方法在现实世界FL应用中努力平衡性能,效率和隐私.
研究的目的:
- 引入一种新的联合学习框架,即动态稀疏联合对比学习 (DSFedCon),旨在解决当前FL方法的局限性.
- 为了提高模型准确性,降低计算成本,并减少FL中的通信开销,特别是对于非IID数据集.
- 在准确性,通信效率和安全性方面评估DSFedCon的有效性和安全性.
主要方法:
- DSFedCon将联合学习与动态稀疏 (DSR) 培训,网络修剪技术和对比学习相结合.
- 该框架旨在优化模型性能,同时最大限度地减少资源利用.
- 在MNIST,CIFAR-10和CIFAR-100数据集上进行了实验,使用不同的迪里克莱特分布参数来模拟非IID数据.
主要成果:
- 与非IID数据集的最先进方法相比,DSFedCon在准确性和通信效率方面表现优越.
- 在通信回合中实现了显著的加快速度:在MNIST上是4.67倍,在CIFAR-10上是7.5倍,在CIFAR-100上是18.33倍,同时保持了可比的训练准确性.
- 分析证实了DSFedCon的通信效率和安全性.
结论:
- DSFedCon提供了一个有效的解决方案,用于非IID数据的联合学习,实现高精度,大幅降低通信成本.
- 拟议的框架代表了高效和私有机器学习的重大进步.
- 对于需要强大的数据安全和隐私的智能系统来说,DSFedCon是一个有前途的方法.
更多相关视频
相关概念视频
Associative Learning
370
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
370
Classification of Systems-II
146
Continuous-time systems have continuous input and output signals, with time measured continuously. These systems are generally defined by differential or algebraic equations. For instance, in an RC circuit, the relationship between input and output voltage is expressed through a differential equation derived from Ohm's law and the capacitor relation,
146
Introduction to Learning
404
Learning is the process of acquiring knowledge or skills through practice or experience, leading to long-lasting behavioral changes. This acquisition occurs through interaction with the environment and requires practice or experience. For instance, mastering a skill such as surfing requires considerable practice and experience, highlighting the essential role of repeated interactions with the environment in learning.
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
404
Classification of Systems-I
186
Linearity is a system property characterized by a direct input-output relationship, combining homogeneity and additivity.
Homogeneity dictates that if an input x(t) is multiplied by a constant c, the output y(t) is multiplied by the same constant. Mathematically, this is expressed as:
Homogeneity dictates that if an input x(t) is multiplied by a constant c, the output y(t) is multiplied by the same constant. Mathematically, this is expressed as:
186
Multi-input and Multi-variable systems
106
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
106
Cognitive Learning
243
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
243


