Neural network for a class of sparse optimization with L₀-regularization.

Area of Science:

• Optimization
• Machine Learning
• Neural Networks

Background:

• Sparse optimization with L0-norm regularization is widely applied but challenging due to nonconvexity and discontinuity.
• Existing methods often struggle with the nonsmooth and discontinuous nature of the L0-norm.

Purpose of the Study:

• To propose a projected neural network modeled by a differential equation for solving sparse optimization problems with L0-norm regularization.
• To address the nonconvex and discontinuous nature of these optimization problems through a smoothing method.

Main Methods:

• A projected neural network modeled by a differential equation is proposed.
• A smoothing method is employed for the L0-norm regularization term to simplify network structure and improve convergence.
• Theoretical analysis proves global existence and uniqueness of solutions, with accumulation points being critical points of the continuous relaxation.

Main Results:

• The proposed network's solutions are globally existent and unique.
• Accumulation points of the network's solutions are critical points of the continuous relaxation model.
• Critical points generally correspond to local minimizers, and all critical points satisfy a promising lower bound property.

Conclusions:

• The developed projected neural network effectively solves sparse optimization problems with L0-norm regularization.
• The smoothing technique enhances convergence properties and network structure.
• Numerical experiments confirm the method's efficiency and performance, especially for feature selection in classification.