Attention-Aware Multi-Task Convolutional Neural Networks.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Multi-task deep learning (MTDL) enhances model generalization by learning multiple tasks concurrently and sharing representations.
• Existing MTDL models often use fixed layer sharing, which may not be optimal for diverse task groups.
• Current methods frequently overlook representation redundancy and lack pre-screening for shared features.

Purpose of the Study:

• To propose an Attention-aware Multi-task Convolutional Neural Network (AMCNN) for automatic and optimal representation sharing in MTDL.
• To address the limitations of fixed layer sharing and the redundancy problem in conventional MTDL approaches.
• To enhance the generalization capabilities and robustness of deep learning models through improved task representation learning.

Main Methods:

• Developed an AMCNN architecture that learns task representation sharing end-to-end.
• Incorporated an attention mechanism to suppress redundant information within shared representations.
• Utilized shortcut connections to preserve essential information during the learning process.

Main Results:

• The proposed AMCNN demonstrated superior performance compared to existing MTDL techniques across various task groups and datasets.
• Experiments confirmed the effectiveness and robustness of the attention mechanism in mitigating representational redundancy.
• The study validated the significant contribution of both the attention mechanism and shortcut connections to the model's overall performance.

Conclusions:

• The AMCNN effectively learns optimal task representation sharing through end-to-end training, outperforming current MTDL methods.
• The integration of attention mechanisms and shortcut connections is crucial for enhancing model generalization and suppressing redundancy.
• This approach offers a more flexible and effective strategy for multi-task learning in deep neural networks.