A Divide-and-Conquer Method for Scalable Robust Multitask Learning.

Area of Science:

• Machine Learning
• Optimization Algorithms

Background:

• Multitask learning (MTL) enhances generalization by leveraging shared task factors.
• Robust MTL (RMTL) uses trace-norm regularization for task relatedness via low-rank structures.
• Existing RMTL algorithms (e.g., APG) are computationally expensive due to repeated Singular Value Decomposition (SVD).

Purpose of the Study:

• To develop a scalable solution for large-scale robust multitask learning (RMTL).
• To address the computational bottleneck of existing RMTL optimization methods.
• To enable efficient RMTL with least squares or squared hinge loss.

Main Methods:

• A divide-and-conquer strategy is proposed to decompose RMTL problems into smaller, parallelizable subproblems.
• Subproblems are solved using base algorithms (e.g., APG) and results are combined.
• Efficient base algorithms utilizing linearized alternating direction methods are developed for specific loss functions.

Main Results:

• The divide-and-conquer RMTL method demonstrates substantial speed improvements over state-of-the-art APG algorithms.
• Theoretical analysis confirms bounded recovery errors, comparable to base algorithms.
• Experimental results show minimal loss in accuracy compared to existing methods.

Conclusions:

• The proposed divide-and-conquer approach offers a computationally efficient and scalable solution for RMTL.
• This method makes advanced RMTL techniques feasible for large-scale, high-dimensional datasets.
• The study advances the practical application of robust multitask learning in machine learning.