lp-lq penalty for sparse linear and sparse multiple kernel multitask learning.

Area of Science:

• Machine Learning
• Optimization
• Computational Biology

Background:

• Multitask learning (MTL) often requires tasks to share a common sparsity profile.
• Existing regularization frameworks induce joint sparsity but lack flexibility.
• Adapting sparsity-inducing penalties is crucial for improved performance and pattern discovery.

Purpose of the Study:

• To investigate the use of generalized l(p)-l(q) mixed norms for joint-sparsity regularization in MTL.
• To develop efficient optimization algorithms for these novel penalties.
• To demonstrate the effectiveness of the proposed methods on various datasets.

Main Methods:

• Derivation of a variational formulation for the l(1)-l(q) penalty in the multiple kernel case.
• Development of an alternating optimization algorithm with guaranteed convergence to the global minimum.
• Extension of accelerated proximal gradient methods for the linear case, including an efficient proximal operator computation.
• Application of a majorization-minimization approach to solve the non-convex problem for general p and q values.

Main Results:

• The proposed alternating optimization algorithm converges globally for the l(1)-l(q) penalized problem.
• An efficient scheme for computing the l(1)-l(q) proximal operator was developed for the linear case.
• The majorization-minimization approach yields an iterative scheme solving weighted l(1)-l(q) sparse MTL problems.
• Empirical results on toy and real-world datasets (BCI, protein localization) validate the benefits of the proposed approaches.

Conclusions:

• Generalized l(p)-l(q) mixed norms offer adaptable and effective sparsity-inducing penalties for multitask learning.
• The developed optimization algorithms are efficient and guarantee convergence.
• The proposed methods demonstrate significant benefits in real-world applications like brain-computer interfaces and protein subcellular localization.