没有偏见的培训辅助任务使初级更好:一个多任务学习的角度.
IEEE transactions on neural networks and learning systems
|September 25, 2024
概括
本研究介绍了一种基于不确定性的多任务学习 (MTL) 公正学习方法. 它确保在所有任务中提供平衡的培训,提高神经网络在主要任务上的性能.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 深度学习 (Deep Learning) 是一种深度学习.
背景情况:
- 多任务学习 (MTL) 利用相关任务的知识来提高主要任务的性能.
- 当前的MTL方法通常通过赋予它们比主要任务更低的损失重量来训练辅助任务.
- 这种不平衡限制了辅助任务在支持主要目标方面的有效性.
研究的目的:
- 为平衡的多任务培训提出基于不确定性的公正学习方法.
- 通过有效利用辅助任务来提高神经网络在主要任务上的性能.
- 为辅助损失制定一个强大的权重策略,以考虑任务不确定性.
主要方法:
- 实施基于不确定性的公正学习方法,以确保平衡的任务培训.
- 在反向传播过程中纳入渐变和不确定性信息,以改善主要任务的重点.
- 开发了一种新的辅助损失权重策略,该策略根据任务不确定性动态调整.
主要成果:
- 拟议的方法实现了与最先进的MTL方法相比或超过的性能.
- 证明有效和强大的增强主要任务的性能.
- 表明权重策略在辅助任务伪标签中对噪声有弹性.
结论:
- 基于不确定性的公正学习为多任务学习提供了平衡和有效的方法.
- 在培训期间考虑任务不确定性和梯度可以显著提高主要任务的性能.
- 拟议的方法提供了一个强大的解决方案,用于利用辅助任务,即使有噪音数据.
相关概念视频
Associative Learning
309
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
309
Real-World Application of Classical Conditioning
537
Classical conditioning not only includes the initial pairing of stimuli but also extends to more complex forms, such as higher-order conditioning. Higher-order conditioning involves creating associations beyond the primary conditioned stimulus, resulting in a chain of conditioned responses.
Higher-order, or second-order, conditioning occurs when a neutral stimulus becomes associated with an already established conditioned stimulus through repeated pairings. For instance, if a dog has been...
Higher-order, or second-order, conditioning occurs when a neutral stimulus becomes associated with an already established conditioned stimulus through repeated pairings. For instance, if a dog has been...
537
Purposive Learning
104
E. C. Tolman emphasized the purposiveness of behavior — the idea that much of our behavior is goal-directed. For instance, employees who aim for a promotion work diligently to meet their targets. Tolman argued that when classical conditioning and operant conditioning occur, the organism acquires certain expectations. In classical conditioning, a child might fear a dog because they expect it to bite. In operant conditioning, a person might consistently work overtime because they expect a...
104
The Anchoring-and-Adjustment Heuristic
7.2K
In order to make good decisions, we use our knowledge and our reasoning. Often, this knowledge and reasoning is sound and solid. However, sometimes, we are swayed by biases or by others manipulating a situation. For example, let’s say you and three friends wanted to rent a house and had a combined target budget of $1,600. The realtor shows you only very run-down houses for $1,600 and then shows you a very nice house for $2,000. Might you ask each person to pay more in rent to get the...
7.2K
Cognitive Learning
229
Cognitive learning is based on purposive behavior, incidental learning, and insight learning.
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
E. C. Tolman's theory of purposive behavior emphasizes that much behavior is goal-directed. He argued that to understand behavior, we must look at the entire sequence of actions leading to a goal. For instance, high school students study hard, not just due to past reinforcement but also to achieve the goal of getting into a good college.
Tolman introduced the idea that behavior is influenced by...
229
Generalization, Discrimination, and Extinction
468
Generalization, discrimination, and extinction are key concepts in operant conditioning that influence how behaviors are learned and maintained.
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
468


