Towards understanding theoretical advantages of complex-reaction networks

Area of Science:

• Artificial Intelligence
• Machine Learning
• Deep Learning

Background:

• Complex-valued neural networks (CVNNs) are gaining traction, but their advantages over real-valued networks require further elucidation.
• Existing research highlights the potential of CVNNs, yet a comprehensive understanding of their comparative benefits is still developing.

Purpose of the Study:

• To introduce and analyze the complex-reaction network, a novel fully-connected feed-forward complex-valued neural network.
• To establish the universal approximation property of complex-reaction networks.
• To compare the parameter efficiency of complex-reaction networks against real-valued networks for specific function approximation tasks.

Main Methods:

• Theoretical analysis to prove the universal approximation property of complex-reaction networks.
• Comparative analysis of parameter requirements for approximating radial functions using complex-reaction networks versus real-valued networks.
• Investigation into the optimization landscape and convergence properties of complex gradient descent for complex-reaction networks.

Main Results:

• The complex-reaction network is proven to possess universal approximation capabilities.
• Approximating radial functions requires polynomial parameters for complex-reaction networks, contrasting with the exponential parameters needed by real-valued networks.
• The critical point set of complex-reaction networks is a proper subset of real-valued networks, suggesting potentially simpler optimization.

Conclusions:

• Complex-reaction networks offer significant advantages in parameter efficiency for approximating certain functions.
• The theoretical findings suggest that complex-reaction networks may facilitate easier identification of optimal solutions due to their simplified critical point landscape.
• This work provides foundational insights into the benefits and properties of complex-valued neural networks, specifically the complex-reaction network architecture.