Online kernel selection for online multi-label classification.

Area of Science:

• Machine Learning
• Data Science
• Computer Vision

Background:

• Online kernel methods are effective for large-scale nonlinear classification.
• Existing methods primarily focus on single-label tasks, with limited options for multi-label classification.
• Current online multi-label kernel methods often use suboptimal offline kernel selection, hindering performance.

Purpose of the Study:

• To develop a novel online multi-label classification approach that unifies kernel selection and model learning.
• To address the challenges of non-convexity in joint optimization for multi-label kernel classifiers.
• To improve the performance of online multi-label classification compared to existing methods.

Main Methods:

• Formulated a joint optimization problem for multi-label kernel classifiers and combination coefficients.
• Proposed an approximation to handle non-convexity, decomposing the problem into efficiently solvable sub-problems.
• Derived a meaningful regret bound for the online learning process.

Main Results:

• The proposed method achieves performance matching the best fixed single-kernel multi-label models.
• Extensive experiments on 11 datasets show superior overall online performance compared to state-of-the-art methods.
• The unified framework effectively integrates kernel selection and incremental model learning.

Conclusions:

• The novel approach offers a significant advancement in online multi-label classification.
• Seamless integration of kernel selection and learning leads to improved performance in dynamic environments.
• The method provides a robust and efficient solution for large-scale multi-label learning tasks.