Multi-Adaptive Optimization for multi-task learning with deep neural networks.

Area of Science:

• Computer Science
• Artificial Intelligence
• Machine Learning

Background:

• Multi-task learning (MTL) leverages task interrelations for deep neural network training.
• Balancing supervisory signals across tasks is a key challenge in MTL.
• Existing task-balancing methods often rely on per-task weighting and may not fully address uneven task contributions.

Purpose of the Study:

• To introduce a novel Multi-Adaptive Optimization (MAO) strategy for MTL.
• To dynamically adjust task contributions to individual network parameters.
• To achieve balanced learning across tasks and parameters automatically.

Main Methods:

• Proposed a novel Multi-Adaptive Optimization (MAO) strategy.
• Implemented MAO to dynamically adjust task contributions to network parameters.
• Conducted comparative experiments on real-world computer vision datasets.

Main Results:

• MAO demonstrated superior performance compared to previous task-balancing alternatives.
• The strategy achieved balanced learning across tasks, network layers, and training steps.
• Experimental analyses provided insights into the advantages of MAO for MTL.

Conclusions:

• MAO offers an effective approach to balancing multi-task learning.
• The dynamic, parameter-specific adaptation of task contributions leads to improved performance.
• This method provides a more comprehensive solution for managing uneven task influence in deep learning.