在头尾不平衡数据上的梯度精细化联合学习
IEEE transactions on neural networks and learning systems
|January 21, 2026
概括
联合学习与不平衡的数据作斗争,其中FedGRE (联合梯度改进) 通过改进梯度来增强全球模型. 这种方法可以提高联合长尾分类中的性能,同时保持数据隐私.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 分布式系统 分布式系统
背景情况:
- 联合学习使分散客户端的协作模式培训能够实现,同时保持数据隐私.
- 一个关键的挑战是联合的长尾学习,由不平衡的数据分布 (头尾不平衡) 造成,这阻碍了模型性能.
- 现有的方法难以平衡阶级知识获取与隐私保护,导致低于最佳的全球模型.
研究的目的:
- 引入FedGRE (联合梯度精制),这是一个新的方法来应对联合长尾学习的挑战.
- 在联合学习中开发一种方法,同时解决数据不平衡,并保持严格的数据隐私.
- 加强在联邦设置中提取类级知识,而不会损害隐私.
主要方法:
- FedGRE使用两个机制构建全球梯度:积累扩散和积累精细化.
- 积累扩散将积累的梯度与随机扰动合并,以减轻阶级失衡.
- 积累精细化使用积累的梯度作为校准的全球更新的,确保一致性和减少振荡. 一种一致性整合技术将精细的梯度纳入全球模型.
主要成果:
- 在6个基准数据集上,FedGRE显著优于14种最先进的方法,用于联合长尾分类.
- 拟议的方法证明了强大的隐私保护以及更好的分类准确性.
- 实验证实了FedGRE在实现阶级平衡的全球优化方面的有效性.
结论:
- 在联邦式学习中,FedGRE有效地解决了类知识获取和隐私保护之间的冲突.
- 在FedGRE中,渐变精细化策略在联合长尾学习场景中带来了卓越的性能.
- 在不平衡的数据集上,FedGRE为构建准确且保护隐私的联合模型提供了一个有前途的解决方案.
相关概念视频
What is an Electrochemical Gradient?
127.4K
Adenosine triphosphate, or ATP, is considered the primary energy source in cells. However, energy can also be stored in the electrochemical gradient of an ion across the plasma membrane, which is determined by two factors: its chemical and electrical gradients.
The chemical gradient relies on differences in the abundance of a substance on the outside versus the inside of a cell and flows from areas of high to low ion concentration. In contrast, the electrical gradient revolves around an...
The chemical gradient relies on differences in the abundance of a substance on the outside versus the inside of a cell and flows from areas of high to low ion concentration. In contrast, the electrical gradient revolves around an...
127.4K
Muscles that Move the Head
5.7K
The muscles that move the head are a dynamic and complex group of structures that work together to facilitate a wide range of head movements, including rotation, flexion, extension, and lateral bending.
The bilateral sternocleidomastoid, or SCM, and the suprahyoid and infrahyoid muscles are significant head flexors. The SCM muscles originate at the sternum and clavicle and attach to the mastoid process of the temporal bone. The SCM contracts bilaterally to bend the head forward, whereas...
The bilateral sternocleidomastoid, or SCM, and the suprahyoid and infrahyoid muscles are significant head flexors. The SCM muscles originate at the sternum and clavicle and attach to the mastoid process of the temporal bone. The SCM contracts bilaterally to bend the head forward, whereas...
5.7K
Avoidance Learning and Learned Helplessness
2.5K
Avoidance learning and learned helplessness are critical concepts in understanding behavioral responses to negative stimuli.
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
2.5K
Energy Line and Hydraulic Gradient Line
2.1K
Based on Bernoulli's equation, the energy line (EL) and hydraulic grade line (HGL) provide graphical representations of energy distribution in a fluid flow system. For steady, incompressible, inviscid flows, Bernoulli's equation is expressed as:
2.1K
Tail-anchoring of Proteins in the ER Membrane
3.7K
Tail-anchored, or TA, proteins are estimated to make up to 3-5% of membrane proteins found in the eukaryotic cell. Such proteins have a single transmembrane domain located approximately 30 amino acid residues upstream from the C-terminal end. As a result, the signal recognition particle (SRP) cannot guide a TA protein to the ER membrane for cotranslational insertion. Hence, they are integrated into the ER membrane post-translationally using their C-terminal end as the anchor. TA proteins...
3.7K
Gradient and Del Operator
4.4K
In mathematics and physics, the gradient and del operator are fundamental concepts used to describe the behavior of functions and fields in space. The gradient is a mathematical operator that gives both the magnitude and direction of the maximum spatial rate of change. Consider a person standing on a mountain. The slope of the mountain at any given point is not defined unless it is quantified in a particular direction. For this reason, a "directional derivative" is defined, which is a vector...
4.4K


