Related Experiment Videos
Neural networks with a continuous squashing function in the output are universal approximators
Summary
Multilayer feedforward networks with output layer squashing functions are proven universal approximators. This corrects a common misapplication of prior research, confirming their function approximation capabilities.
Area of Science:
- Artificial Intelligence
- Machine Learning
- Neural Networks
Background:
- Prior work by Hornik and Funahashi (1989) established universal approximation for feedforward networks lacking output layer squashing functions.
- This foundational result has been frequently misapplied to networks that *do* include output layer squashing functions.
Discussion:
- This paper rigorously proves that multilayer feedforward neural networks *with* squashing functions in the output layer are also universal approximators.
- The study addresses and rectifies the improper application of previous findings in neural network theory.
Key Insights:
- Neural networks with output squashing functions can approximate any Borel measurable function to arbitrary precision.
- Sufficiently many hidden units are the key requirement for achieving this universal approximation capability.
Outlook:
- This finding has significant implications for the theoretical understanding and practical application of neural networks in function approximation.
- It validates the use of a broader class of neural network architectures for complex modeling tasks.