Two Recurrent Neural Networks With Reduced Model Complexity for Constrained l₁-Norm Optimization

Area of Science:

• Optimization
• Machine Learning
• Neural Networks

Background:

• Least Absolute Deviation (LAD or l1) optimization offers robustness and sparsity.
• Recurrent Neural Networks (RNNs) can solve constrained l1-norm problems but have slow convergence.
• Efficient solution methods for constrained l1-norm optimization are crucial.

Purpose of the Study:

• To develop novel RNNs for accelerated convergence in constrained l1-norm optimization.
• To introduce continuous- and discrete-time RNN systems for solving l1-norm optimization with linear constraints.
• To theoretically prove global convergence of the proposed RNNs.

Main Methods:

• Development of two novel RNNs: a continuous-time system and a discrete-time system.
• Formulation of RNNs to address l1-norm optimization with linear equality and inequality constraints.
• Theoretical analysis to prove global convergence to optimal solutions without conditions.

Main Results:

• The proposed RNNs achieve global convergence to optimal solutions unconditionally.
• The new RNNs exhibit significantly reduced model complexity compared to existing methods.
• Numerical simulations confirm substantially faster computational times than related RNNs and traditional algorithms.

Conclusions:

• The introduced continuous- and discrete-time RNNs effectively and rapidly solve constrained l1-norm optimization problems.
• These RNNs offer a significant improvement in convergence speed for LAD optimization.
• The methods provide a computationally efficient alternative for linearly constrained l1-norm optimization.