Related Experiment Video
Updated: Jul 7, 2026

Closed-loop Neuro-robotic Experiments to Test Computational Properties of Neuronal Networks
Published on: March 2, 2015
Learning efficiency of redundant neural networks in Bayesian estimation
1Precision and Intelligence Laboratory, Tokyo Institute of Technology, Yokohama 226-8503, Japan. swatanab@pi.titech.ac.jp
Bayesian stochastic complexity in layered neural networks is asymptotically smaller than in regular statistical models when the true distribution is present. This finding suggests improved generalization error for neural networks in such scenarios.
Area of Science:
- Machine Learning
- Statistical Modeling
- Information Theory
Background:
- Layered neural networks (LNNs) are powerful function approximators.
- Understanding their statistical properties compared to traditional models is crucial.
- Bayesian methods offer a principled approach to model complexity and generalization.
Purpose of the Study:
- To compare the Bayesian stochastic complexity of LNNs with regular statistical models.
- To analyze the generalization error of LNNs under specific conditions.
- To elucidate the statistical differences between LNNs and traditional models.
Main Methods:
- Asymptotic analysis of Bayesian stochastic complexity.
- Theoretical comparison using a three-layer perceptron model.
- Derivation of bounds on generalization error.
Main Results:
- Bayesian stochastic complexity of LNNs is asymptotically smaller than regular models when the true distribution is included.
- A specific bound for a three-layer perceptron is derived: (1/2) {H(0) (M+N)+R} log n.
- Generalization error is shown to be smaller, related to the increase in stochastic complexity.
Conclusions:
- LNNs exhibit favorable statistical complexity properties compared to regular models under certain conditions.
- The derived bounds provide theoretical support for the effectiveness of LNNs in capturing true distributions.
- This work offers a statistical perspective on the advantages of LNNs for generalization.
Related Concept Videos
One-Compartment Open Model: Wagner-Nelson and Loo Riegelman Method for ka Estimation
On...
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
Propagation of Uncertainty from Random Error
Propagation of Uncertainty from Systematic Error
Survival Tree
Building a Survival Tree
Constructing a survival tree begins...