Top-$k$k Feature Selection in Sparse Learning via Accelerated Coordinate Descent Method.

Area of Science:

• Machine Learning
• Sparse Learning
• Optimization

Background:

• Top-k feature selection is crucial in sparse learning but challenging due to the l2,0-norm constraint.
• Existing methods often relax this constraint, leading to suboptimal solutions and degraded models.

Purpose of the Study:

• To address the limitations of current top-k feature selection methods in sparse learning.
• To develop a unified approach for both supervised and semi-supervised top-k feature selection.
• To propose an efficient algorithm for solving the underlying non-convex optimization problem.

Main Methods:

• Unified two distinct objectives into a single ratio-trace non-convex optimization problem.
• Developed an accelerated coordinate descent method to efficiently solve the non-convex objective.
• Investigated the universality of the approach across supervised and semi-supervised learning.

Main Results:

• Obtained local optimal solutions for top-k feature indices with competitive time complexity.
• Demonstrated the advantages of selected features through visualized toy experiments.
• Achieved superior performance compared to state-of-the-art methods on nine benchmark datasets and ImageNet.

Conclusions:

• The proposed accelerated coordinate descent method effectively solves the generalized ratio-trace problem for top-k feature selection.
• The unified approach offers a robust and efficient solution for both supervised and semi-supervised sparse learning scenarios.
• Experimental results validate the superiority and effectiveness of the developed algorithm over existing methods.