Multi-task neural networks by learned contextual inputs.

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Multi-task learning (MTL) architectures often require complex parameter sharing strategies.
• Adapting models to new tasks or data can be computationally intensive.
• Existing architectures may struggle with datasets containing limited data points per task.

Purpose of the Study:

• To introduce and evaluate a novel multi-task learning architecture: learned-context neural networks.
• To demonstrate the efficacy of a powerful task adaptation mechanism.
• To investigate the properties and benefits of a low-dimensional task parameter space.

Main Methods:

• Developed a fully shared neural network architecture augmented with trainable task parameters.
• Theoretically analyzed the universal approximation capabilities with a scalar task parameter.
• Empirically validated the architecture's performance on ten diverse datasets.

Main Results:

• The proposed architecture facilitates a low-dimensional task parameter space, proving theoretically sufficient for universal approximation.
• Empirical results show a small task parameter space is viable, with dimension potentially varying based on task complexity.
• Demonstrated robustness on datasets with few data points per task and simplified model updating workflows.

Conclusions:

• Learned-context neural networks provide an effective and efficient approach to multi-task learning.
• The architecture's task parameter space is well-behaved, simplifying adaptation and learning of new tasks.
• Achieved competitive performance compared to existing neural network architectures.