A convex formulation for learning a shared predictive structure from multiple tasks.

Area of Science:

• Machine Learning
• Computer Science
• Data Science

Background:

• Multitask learning aims to improve generalization by leveraging shared structures across related tasks.
• Existing alternating structure optimization (ASO) algorithms are effective but suffer from nonconvexity, leading to local optima.
• Scalability remains a challenge for traditional methods when dealing with large datasets.

Purpose of the Study:

• To develop novel multitask learning formulations that address the nonconvexity and local optima issues of ASO.
• To introduce a convex relaxation (rASO) with theoretical guarantees of global optimality.
• To propose efficient algorithms for solving the relaxed convex formulation on large-scale datasets.

Main Methods:

• Introduced an improved ASO formulation (iASO) with a new regularizer.
• Developed a convex relaxation (rASO) of iASO, theoretically shown to achieve global optimum under specific conditions.
• Employed block coordinate descent (BCD) and accelerated projected gradient (APG) algorithms for efficient optimization of rASO.

Main Results:

• Demonstrated that the proposed rASO formulation can find globally optimal solutions.
• Developed efficient algorithms for key subproblems within BCD and APG, enhancing computational efficiency.
• Experimental validation on Yahoo webpages and Drosophila gene expression datasets confirmed the effectiveness and efficiency of the proposed methods.

Conclusions:

• The proposed iASO and rASO formulations effectively address limitations of existing multitask learning algorithms.
• The developed BCD and APG-based methods provide scalable and efficient solutions for finding globally optimal multitask learning parameters.
• The study confirms the theoretical analysis through successful experimental application on diverse real-world datasets.