SparseNet: Coordinate Descent With Nonconvex Penalties

Area of Science:

• Machine Learning
• Statistical Modeling
• Optimization Algorithms

Background:

• Sparse selection in linear models is crucial for interpretability and feature selection.
• Existing methods often rely on convex relaxation or complex algorithms.
• Nonconvex penalties offer potential for improved sparsity but pose optimization challenges.

Purpose of the Study:

• To develop and analyze a coordinate-descent approach for sparse selection in linear models.
• To characterize suitable nonconvex penalties and their threshold functions for coordinate-descent.
• To demonstrate the efficacy of the proposed method using the MC+ penalty.

Main Methods:

• Coordinate-descent optimization algorithm applied to linear models.
• Characterization of penalty properties and threshold functions.
• Development of a df-standardizing reparametrization for pathwise algorithms.
• Application and evaluation using the MC+ penalty.

Main Results:

• The coordinate-descent approach demonstrates convergence properties for sparse selection.
• Specific penalty characteristics suitable for this optimization strategy are identified.
• The df-standardizing reparametrization facilitates a pathwise algorithm.
• The MC+ penalty proves well-suited for the proposed coordinate-descent method.

Conclusions:

• Coordinate-descent offers an effective optimization strategy for sparse linear models.
• The MC+ penalty is a suitable choice for this approach, enhancing performance.
• The study provides theoretical insights and practical demonstration of the algorithm.