Architecture-independent approximation of functions.

Area of Science:

• Machine Learning
• Artificial Intelligence
• Neural Networks

Background:

• Feedforward networks are fundamental in machine learning.
• Understanding network size and complexity is crucial for performance.
• Weight distribution impacts network behavior.

Purpose of the Study:

• To investigate the relationship between error minimization and network size.
• To determine how weight distribution affects function complexity.
• To identify methods for controlling neural network complexity.

Main Methods:

• Minimizing expected error over a distribution of weights.
• Analyzing the impact of weight variance on function complexity.
• Examining network behavior as the number of hidden units increases.

Main Results:

• Approximation becomes independent of network size with growing hidden units.
• Function complexity is regulated by weight distribution variance.
• Complexity stabilizes or changes minimally above a certain network size.

Conclusions:

• Network complexity is primarily controlled by weight variance, not architecture.
• Sufficient hidden units allow variance to dictate function complexity.
• This offers a method for predictable complexity control in feedforward networks.