Accommodating Multiple Tasks' Disparities With Distributed Knowledge-Sharing Mechanism.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Deep multitask learning (MTL) leverages shared knowledge to improve performance, especially in data-scarce scenarios.
• Existing MTL methods struggle with tasks having different complexities and input sizes, such as processing RGB and grayscale images simultaneously.
• Heterogeneous network architectures and varied input dimensions pose significant challenges for effective knowledge sharing in MTL.

Purpose of the Study:

• To introduce a novel framework, Tensor Ring Multitask Learning (TRMTL), designed to address the limitations of current MTL approaches.
• To enable flexible knowledge sharing across heterogeneous networks and tasks with diverse input sizes within an MTL setting.
• To enhance the performance of data-insufficient tasks by effectively managing disparities in network design and input data.

Main Methods:

• Developed a distributed knowledge-sharing framework based on tensor ring decomposition.
• Decoupled the relationship between knowledge sharing and original weight matrices.
• Designed TRMTL to accommodate heterogeneous networks and tasks with varied input sizes.

Main Results:

• TRMTL demonstrated significant improvements in performance, particularly for data-insufficient tasks.
• The framework proved effective in handling disparities in network complexity and input sizes across different tasks.
• Experimental validation confirmed the effectiveness, efficiency, and flexibility of TRMTL on challenging datasets.

Conclusions:

• TRMTL offers a flexible and effective solution for multitask learning with heterogeneous tasks and data.
• The proposed method successfully mitigates performance degradation caused by data scarcity and input variations.
• TRMTL represents a significant advancement in enabling robust knowledge sharing across diverse deep learning tasks.