Task's Choice: Pruning-Based Feature Sharing (PBFS) for Multi-Task Learning.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Computer Science

Background:

• Existing multi-task learning (MTL) models often use parameter sharing (hard, soft, hierarchical) for public information.
• Model pruning offers potential for knowledge transfer via sparse sharing, but struggles with conflicting tasks and private information learning.
• Inefficiencies in current MTL methods include inadequate private information learning and negative knowledge transfer.

Purpose of the Study:

• To propose a novel multi-task learning model, Pruning-Based Feature Sharing (PBFS).
• To address limitations of existing MTL models in handling conflicting tasks and learning private information.
• To improve knowledge transfer and overall model performance in multi-task scenarios.

Main Methods:

• Developed the Pruning-Based Feature Sharing (PBFS) model.
• Integrated a soft parameter sharing structure with model pruning.
• Introduced a prunable shared network connecting task-specific subnets, allowing tasks to select parameters as needed.

Main Results:

• Experiments on benchmark and synthetic datasets demonstrated the effectiveness of the proposed information sharing strategy.
• Analyzed the impact of subnet sparsity and task correlations on model performance.
• The PBFS model showed superior performance compared to several existing multi-task learning models.

Conclusions:

• The proposed pruning-based feature sharing strategy is beneficial for transfer learning in multi-task scenarios.
• PBFS effectively addresses challenges posed by conflicting tasks and enhances the learning of private task information.
• The model demonstrates superiority over traditional multi-task learning approaches, particularly in complex task relationships.