Parameterized Convex Universal Approximators for Decision-Making Problems.

Area of Science:

• Optimization
• Machine Learning
• Convex Analysis

Background:

• Existing convex approximators like max-affine (MA) and log-sum-exp (LSE) networks have limitations in general decision-making.
• Generalizing these networks requires incorporating condition and decision variables and parameterizing network weights.

Purpose of the Study:

• To introduce and analyze parameterized max-affine (PMA) and parameterized log-sum-exp (PLSE) networks.
• To prove the universal approximation capabilities of PMA and PLSE for parameterized convex continuous functions.
• To provide practical guidelines for integrating deep neural networks into PMA and PLSE architectures.

Main Methods:

• Theoretical analysis to prove the universal approximation theorem (UAT) for PMA and PLSE networks.
• Development of novel network architectures that generalize MA and LSE by using continuous functions for parameters.
• Numerical simulations to evaluate the performance of PMA and PLSE against existing methods.

Main Results:

• The universal approximation theorem (UAT) is proven for PMA and PLSE, establishing them as shape-preserving universal approximators.
• Practical guidelines for deep neural network integration are provided.
• Simulation results indicate that PLSE networks outperform existing approximators in minimizing errors for both minimizer and optimal value.

Conclusions:

• PMA and PLSE networks are effective universal approximators for parameterized convex continuous functions.
• PLSE networks offer a scalable and computationally efficient solution for high-dimensional decision-making problems.
• The proposed networks represent a significant advancement in convex approximation for machine learning and optimization.