针对联合自主监督学习的特征相关指导知识转移
概括
联合自主监督学习 (FedSSL) 通过交换特征相关性而不是参数或特征来克服标签稀缺性. 这种新的方法,Federated FoA,使异质客户之间的协作成为可能,改善了模型性能.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 分布式系统 分布式系统
背景情况:
- 自主监督学习 (SSL) 应用于联合学习 (FL),以应对数据标签挑战.
- 现有的联合SSL (FedSSL) 方法通常假设均模型或需要公开数据集,从而限制了它们的一般适用性.
- 不同质的模型和未标记的客户端对通用FedSSL框架构成重大障碍.
研究的目的:
- 提出一种新且通用的方法,即以特征相关性为基础的聚合 (FedFoA) 的联合自我监督学习,用于在异质的联合环境中进行培训.
- 克服现有的FedSSL方法的局限性,这些方法依赖于参数或特征共享.
- 为了实现有效的知识转移和协作在异质模型的未标记的客户之间.
主要方法:
- FedFoA 交换功能相关性,而不是模型参数或功能映射,以减少局部表示学习中的差异.
- 一种基于因数分解的方法从局部表示中提取一个跨特征关系矩阵,作为聚合的知识介质.
- 该框架旨在支持异质性,保护隐私,并与现有的FedSSL方法兼容.
主要成果:
- 实际上,FedFoA有效地减少了地方代表学习过程中的差异.
- 拟议的方法促进了异质客户之间的合作.
- 广泛的实验表明,FedFoA显著超过了最先进的方法.
结论:
- FedFoA为联合自主监督学习提供了通用和有效的解决方案,特别是在异质的环境中.
- 基于特征相关性的聚合方法可以提高协作和性能,而无需对客户端模型做出强有力的假设.
- 该方法在现有方法上显示了显著的改进,突出了其对现实世界的应用的潜力.
更多相关视频
相关概念视频
Associative Learning
276
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
276
Improving Translational Accuracy
2.5K
2.5K
Self-Schemas
30.9K
In general, a schema is a mental construct consisting of a cluster or collection of related concepts (Bartlett, 1932). There are many different types of schemata, and they all have one thing in common: schemata are a method of organizing information that allows the brain to work more efficiently. When a schema is activated, the brain makes immediate assumptions about the person or object being observed.
30.9K
Generalization, Discrimination, and Extinction
399
Generalization, discrimination, and extinction are key concepts in operant conditioning that influence how behaviors are learned and maintained.
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
399
Correlation and Regression
1.2K
In statistics, correlation describes the degree of association between two variables. In the subfield of linear regression, correlation is mathematically expressed by the correlation coefficient, which describes the strength and direction of the relationship between two variables. The coefficient is symbolically represented by 'r' and ranges from -1 to +1. A positive value indicates a positive correlation where the two variables move in the same direction. A negative value suggests a...
1.2K
Correlation
11.5K
In statistics, two variables are said to be correlated if the values of one variable are associated with the other variable. Depending on the relationship between two variables, correlation can be of three types– positive correlation, negative correlation, and zero correlation.
Two variables, for example, a and b, are said to be positively correlated if both variables move in the same direction. In other words, a positive correlation exists between two variables, a and b, if:
Two variables, for example, a and b, are said to be positively correlated if both variables move in the same direction. In other words, a positive correlation exists between two variables, a and b, if:
11.5K


