在多输出DNN中学习任务首选的推理路线以缓解梯度冲突.
IEEE transactions on pattern analysis and machine intelligence
|November 24, 2025
概括
多输出神经网络 (MON) 遭受任务干扰. DR-MGF (动态路线和超权重梯度融合) 学习了特定任务的波器的重要性,以创建动态路线,减少干扰和提高性能.
科学领域:
- 人工智能的人工智能
- 机器学习 机器学习
- 深度学习 (Deep Learning) 是一种深度学习.
背景情况:
- 多输出神经网络 (MON) 具有跨任务的共享过器,导致纠的推理路线.
- 在MON中不同的优化目标会在共享路线上造成任务梯度干扰,阻碍模型的整体性能.
研究的目的:
- 提出一种新的梯度脱冲突算法,DR-MGF (动态路线和超权重梯度融合),用于MONs.
- 通过学习任务优先推断路径来解决MON中任务间干扰的问题.
主要方法:
- DR-MGF学习特定任务的重要变量,以评估不同任务的波器重要性.
- 任务对过器的主导地位与特定任务的过器重要性成比例调整,减少任务间干扰.
- 任务特定的重要变量动态地确定任务首选的推断路线.
主要成果:
- 在MONs中,DR-MGF有效地减少了任务间干扰.
- 在CIFAR,ImageNet和NYUv2数据集上的实验结果显示,DR-MGF的性能优于现有的脱冲突方法.
- 拟议的DR-MGF方法可以在没有结构修改的情况下扩展到一般的MON.
结论:
- DR-MGF提供了一种有效的解决方案,可以在MONs中消除梯度冲突.
- 动态路线学习方法通过减轻任务干扰来显著提高MON性能.
- DR-MGF提供了一种灵活和可通用的方法来增强多输出深度学习模型.
相关概念视频
Multi-input and Multi-variable systems
376
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence of...
In the absence of...
376
Associative Learning
1.2K
Associative learning is a fundamental concept in behavioral psychology, wherein a connection is established between two stimuli or events, leading to a learned response. This process is critical in understanding how behaviors are acquired and modified. Conditioning, the mechanism through which associations are formed, can be divided into two main types: classical conditioning and operant conditioning, each elucidating different aspects of associative learning.
Classical conditioning, also known...
Classical conditioning, also known...
1.2K
Neural Circuits
2.6K
Neural circuits and neuronal pools are two of the main structures found in the nervous system. Neural circuits are networks of neurons that work together to carry out a specific task or process. They consist of interconnected neurons and glial cells, which provide structural and metabolic support.
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
2.6K
Gradient and Del Operator
4.3K
In mathematics and physics, the gradient and del operator are fundamental concepts used to describe the behavior of functions and fields in space. The gradient is a mathematical operator that gives both the magnitude and direction of the maximum spatial rate of change. Consider a person standing on a mountain. The slope of the mountain at any given point is not defined unless it is quantified in a particular direction. For this reason, a "directional derivative" is defined, which is a vector...
4.3K
Observational Learning
802
Albert Bandura's observational learning, also known as imitation or modeling, occurs when a person observes and imitates another's behavior. It is a quicker process than operant conditioning. A well-known example is the Bobo doll study, where children who saw an adult acting aggressively towards the doll were more likely to act aggressively when left alone, compared to those who observed a nonaggressive adult. Many psychologists view observational learning as a form of latent learning...
802
Avoidance Learning and Learned Helplessness
2.5K
Avoidance learning and learned helplessness are critical concepts in understanding behavioral responses to negative stimuli.
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
Avoidance learning occurs when an organism learns that a specific behavior can prevent an unpleasant outcome. For example, a student who receives a bad grade may start studying harder to avoid future poor grades. This behavior persists even when the negative outcome is no longer present. Avoidance learning is powerful because it maintains behavior in the absence of the...
2.5K

