专家组合个性化:为视觉语言模型提供联合的即时学习
Jun Luo1, Chen Chen2, Shandong Wu3
1Intelligent Systems Program, University of Pittsburgh, Pittsburgh, PA 15213, USA.
... International Conference on Learning Representations
|February 16, 2026
概括
自适应提示的个性化联合混合 (pFedMoAP) 通过允许客户使用多个非本地提示来增强联合学习. 这种方法可以改善视觉语言模型个性化和跨不同数据集的性能.
科学领域:
- 计算机科学 计算机科学
- 人工智能的人工智能
- 机器学习 机器学习
背景情况:
- 联合学习 (FL) 通常将客户限制在一个单一的全球模型中,限制个性化.
- 类似于CLIP的视觉语言模型 (VLM) 通过快速学习提供了强大的表示学习.
- 现有的联合快速学习方法继承了传统FL范式的局限性.
研究的目的:
- 提出一种新的联合提示学习框架,即适应提示的个性化联合混合 (pFedMoAP).
- 为了解决传统的FL在处理VLM轻量级提示的局限性.
- 通过使客户能够利用多个预先聚合的提示来增强联合学习中的个性化.
主要方法:
- 介绍了pFedMoAP,这是一个联合的学习框架,利用专家混合 (MoE) 方法进行快速学习.
- 方便客户下载多个预先汇总的提示,作为固定非本地专家.
- 实施了基于注意力的本地门网,以实现个性化的快速适应和增强的文本功能生成.
主要成果:
- 在各种联合设置下,在9个不同的数据集中证明了pFedMoAP的有效性.
- 通过自适应提示专家展示了本地图像数据和增强的文本功能之间的改进对齐.
- 在联邦快速学习中验证了个性化的好处.
结论:
- pFedMoAP通过将非本地专家纳入专家混合框架,有效地个性化联合快速学习.
- 拟议的方法在联合环境中增强了VLM表示学习,优于传统方法.
- 该框架提供了一个灵活和高效的解决方案,用于个性化的联合学习,使用轻量级提示.
相关概念视频
01:12Observational Learning
1.0K
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
1.0K
01:27Associative Learning
1.5K
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
1.5K
01:18Introduction to Learning
1.2K
Learning is the process of acquiring knowledge or skills through practice or experience, leading to long-lasting behavioral changes. This acquisition occurs through interaction with the environment and requires practice or experience. For instance, mastering a skill such as surfing requires considerable practice and experience, highlighting the essential role of repeated interactions with the environment in learning.
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
1.2K
01:21Cognitive Learning
1.4K
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
1.4K
01:22Purposive Learning
531
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
531
01:22Multi-input and Multi-variable systems
431
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence of...
In the absence of...
431