Bayesian learning of feature spaces for multitask regression.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Computational Statistics

Background:

• Multi-task learning (MTL) aims to improve generalization by learning multiple related tasks simultaneously.
• Constrained architecture complexity is crucial for efficient and scalable machine learning models.
• Kernel methods offer robustness but can be computationally expensive.

Purpose of the Study:

• To introduce a novel multi-task regression model, RFF-BLR, with constrained architecture complexity.
• To leverage random Fourier features (RFF) for approximating kernel methods within a neural network.
• To employ a Bayesian formulation for optimizing model weights and promoting sparsity.

Main Methods:

• The proposed RFF-BLR model utilizes a single hidden layer with RFF units approximating a Radial Basis Function (RBF) kernel.
• A Bayesian approach optimizes weights, enabling tasks to interact during training.
• Multi-output sparsity is promoted by selecting a compact subset of hidden units for common non-linear mapping.

Main Results:

• RFF-BLR demonstrates significant performance improvements over state-of-the-art methods in multi-task nonlinear regression.
• The model excels particularly in scenarios with small-sized training datasets.
• The RFF-based hidden layer provides robustness, characteristic of kernel methods.

Conclusions:

• The RFF-BLR framework offers an effective approach for multi-task nonlinear regression with controlled complexity.
• The combination of RFF and Bayesian optimization enables efficient learning and feature selection.
• This method shows promise for applications requiring robust performance on limited data.