Related Experiment Video
Updated: May 1, 2026

Modeling the Functional Network for Spatial Navigation in the Human Brain
Published on: October 13, 2023
Universal approximation depth and errors of narrow belief networks with discrete units
1Department of Mathematics, Pennsylvania State University, University Park, PA 16802, U.S.A. montufar@mis.mpg.de.
Abstract:
We generalize recent theoretical work on the minimal number of layers of narrow deep belief networks that can approximate any probability distribution on the states of their visible units arbitrarily well. We relax the setting of binary units (Sutskever & Hinton, 2008 ; Le Roux & Bengio, 2008 , 2010 ; Montúfar & Ay, 2011 ) to units with arbitrary finite state spaces and the vanishing approximation error to an arbitrary approximation error tolerance. For example, we show that a q-ary deep belief network with L > or = 2 + (q[m-delta]-1 / (q-1)) layers of width n < or = + log(q) (m) + 1 for some [Formula : see text] can approximate any probability distribution on {0, 1, ... , q-1}n without exceeding a Kullback-Leibler divergence of delta. Our analysis covers discrete restricted Boltzmann machines and naive Bayes models as special cases.
Related Concept Videos
Propagation of Uncertainty from Random Error
Accuracy, limits, and approximation
Accuracy is defined as the closeness of the measured value to the true or actual value. In engineering mechanics, repeated measurements are taken during theoretical or experimental analyses to ensure that the result is precise and accurate.
The accuracy of any solution is based on the...
Propagation of Uncertainty from Systematic Error
Uncertainty: Confidence Intervals
Application of Linearization and Approximation
Linearization and Approximation
