Smoothing neural network for L₀ regularized optimization problem with general convex constraints.

Area of Science:

• Computational Mathematics
• Machine Learning
• Optimization Theory

Background:

• Sparse regression problems often involve non-convex and discontinuous objective functions, posing significant challenges for traditional optimization methods.
• Existing neural network approaches for non-smooth, non-convex problems can be complex and lack guaranteed convergence properties.

Purpose of the Study:

• To propose a novel neural network model capable of solving discontinuous and non-convex sparse regression problems.
• To address challenges associated with L0 regularization in optimization.
• To analyze the convergence and solution properties of the proposed neural network.

Main Methods:

• A neural network model based on differential inclusion is developed.
• A smoothing relaxation function is constructed for the L0 regularization term.
• Theoretical analysis is employed to prove the existence, boundedness, and finite-time convergence of the network's solution.

Main Results:

• The proposed neural network guarantees global existence, boundedness, and convergence to the feasible region for solutions starting from any point satisfying linear equality constraints.
• Accumulation points of the solution are identified as Clarke stationary points of the smoothed approximation problem.
• In box-constrained cases, accumulation points exhibit a unified lower bound and common support set, with most being local minimizers.

Conclusions:

• The proposed neural network offers a simpler structure compared to existing methods for solving non-smooth, non-convex problems.
• Numerical experiments validate the efficiency and effectiveness of the developed neural network for sparse regression.
• The theoretical guarantees provide strong evidence for the robustness of the approach.