识别多任务学习的任务分组,使用点位V可用信息
Yingya Li1, Timothy Miller1, Steven Bethard2
1Computational Health Informatics Program, Boston Children's Hospital, and Harvard Medical School, 401 Park Drive, Boston, MA 02115, USA.
Journal of biomedical informatics
|July 18, 2025
概括
一个使用点位V可用信息 (PVI) 的新指标有助于组化多任务学习的任务. 这种方法提高了微调效率和性能,优于语言模型的随机分组.
科学领域:
- 自然语言处理自然语言处理.
- 机器学习 机器学习
背景情况:
- 微调大型语言模型 (LLM) 对效率和性能至关重要.
- 多任务学习可以提高绩效,但对任务分组很敏感,风险是负面转移.
- 确定最佳的任务分组仍然是一个挑战.
研究的目的:
- 提出一种用于量化任务相关性的新型指标,以指导多任务学习分组.
- 为了评估这个指标在各种NLP数据集中的有效性.
主要方法:
- 任务相关性是使用点位V可用信息 (PVI) 来衡量的,这是数据集信息内容的指标.
- 具有统计上相似的PVI估计的任务被分组为联合学习.
- 对15个NLP数据集进行了实验,涉及到一般,生物医学和临床领域,与Llama和GPT-4等单任务模型和LLM进行了比较.
主要成果:
- 分组具有相似PVI估计的任务导致联合学习者实现竞争性表现.
- 与其他方法相比,这些联合学习者使用的总参数较少.
- 在不同领域观察到一致的性能.
结论:
- 基于PVI的任务分组指标为多任务学习提供了一个有益的方法,特别是在特定领域的应用程序中.
- 微调模型仍然是一个强大的选择,这个指标可以增强他们的微调策略.
- 拟议的指标可以整合到LLMs更广泛的微调方法中.
相关概念视频
Associative Learning
586
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
586
Multi-input and Multi-variable systems
150
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
In the absence...
150
Purposive Learning
207
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
207
Normal and Tangetial Components: Problem Solving
227
Consider a man with a mass of 70 kg seated in a chair connected to a pin support through a member BC. If the man maintains an upright position, the task is to determine the horizontal and vertical reactions of the chair on the man when the member makes a 45° angle with the horizontal. At this moment, the man has a speed of 5 m/s, increasing at a rate of 1 m/s².
227
Multicompartment Models: Overview
258
Multicompartment models are mathematical constructs that depict how drugs are distributed and eliminated within the body. They segment the body into several compartments, symbolizing various physiological or anatomical areas connected through drug transfer processes such as absorption, metabolism, distribution, and elimination.
These models offer a more comprehensive representation of drug behavior in the body than one-compartment models. They accommodate the complexity of drug distribution,...
These models offer a more comprehensive representation of drug behavior in the body than one-compartment models. They accommodate the complexity of drug distribution,...
258
Observational Learning
317
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
317


