Design of recurrent neural networks for solving constrained least absolute deviation problems

Area of Science:

• Computational Neuroscience
• Machine Learning
• Optimization Theory

Background:

• Recurrent neural networks (RNNs) are increasingly used for L(1)-norm optimization, specifically least absolute deviation (LAD) problems.
• Existing RNNs often struggle with complex linear constraints, limiting their applicability.
• There is a need for robust neural network architectures capable of handling various constraint types in LAD problems.

Purpose of the Study:

• To propose novel recurrent neural network architectures for solving constrained least absolute deviation (LAD) problems.
• To address limitations of existing networks by enabling the handling of diverse linear constraints, including equality, two-sided inequality, and bound constraints.
• To develop specialized networks for specific constraint combinations, such as equality and one-sided inequality constraints.

Main Methods:

• Design and theoretical analysis of two new recurrent neural network architectures for general constrained LAD problems.
• Development of a specialized network for LAD problems with both equality and one-sided inequality constraints.
• Rigorous mathematical proofs demonstrating the capability of the proposed networks to solve the target optimization problems.

Main Results:

• The proposed networks effectively solve LAD problems with equality, two-sided inequality, and bound constraints.
• Networks tailored for specific constraint subsets offer simpler architectures compared to existing methods.
• Networks designed for the same problem types exhibit identical structural complexity, facilitating flexible circuit implementation.

Conclusions:

• The developed recurrent neural networks provide a powerful and flexible framework for solving a wide range of constrained L(1)-norm optimization problems.
• The theoretical results are validated through numerical simulations, demonstrating convergence and comparing performance.
• The architectural flexibility enhances practical applicability in hardware implementations for LAD problem solving.