On the Kolmogorov neural networks

Area of Science:

• Artificial Intelligence
• Machine Learning
• Neural Networks

Background:

• Multivariate function approximation is a fundamental problem in computational mathematics.
• Neural networks are powerful tools for function approximation, but their representational capacity varies with architecture and activation functions.

Purpose of the Study:

• To investigate the representational capabilities of the Kolmogorov two hidden layer neural network model.
• To determine the impact of different activation functions in the second hidden layer on the model's ability to approximate various types of multivariate functions.

Main Methods:

• Utilizing the Kolmogorov two hidden layer neural network architecture.
• Employing continuous, discontinuous bounded, and unbounded activation functions in the second hidden layer.
• Analyzing the precise representation of continuous, discontinuous bounded, and unbounded multivariate functions.

Main Results:

• The Kolmogorov two hidden layer neural network precisely represents continuous multivariate functions when using a continuous activation function.
• The model accurately represents discontinuous bounded multivariate functions with a discontinuous bounded activation function.
• The network effectively approximates all unbounded multivariate functions using an unbounded activation function.

Conclusions:

• The Kolmogorov two hidden layer neural network model demonstrates universal approximation capabilities for a wide range of multivariate functions.
• The choice of activation function in the second hidden layer is critical for determining the specific class of functions the network can represent.
• This research provides theoretical insights into the expressive power of specific neural network architectures.