Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Survival Tree01:19

Survival Tree

294
Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
 Building a Survival Tree
Constructing a...
294
Generalization, Discrimination, and Extinction01:24

Generalization, Discrimination, and Extinction

1.2K
Generalization, discrimination, and extinction are key concepts in operant conditioning that influence how behaviors are learned and maintained.
Generalization occurs when a behavior reinforced in one context is performed in similar situations. For instance, a student who studies diligently for calculus and receives excellent grades might apply the same study habits to psychology and history, expecting similar results. Generalization shows how learning in one setting can influence behavior in...
1.2K
Propagation of Uncertainty from Systematic Error01:10

Propagation of Uncertainty from Systematic Error

1.2K
The atomic mass of an element varies due to the relative ratio of its isotopes. A sample's relative proportion of oxygen isotopes influences its average atomic mass. For instance, if we were to measure the atomic mass of oxygen from a sample, the mass would be a weighted average of the isotopic masses of oxygen in that sample. Since a single sample is not likely to perfectly reflect the true atomic mass of oxygen for all the molecules of oxygen on Earth, the mass we obtain from this...
1.2K
Propagation of Uncertainty from Random Error00:59

Propagation of Uncertainty from Random Error

1.5K
An experiment often consists of more than a single step. In this case, measurements at each step give rise to uncertainty. Because the measurements occur in successive steps, the uncertainty in one step necessarily contributes to that in the subsequent step. As we perform statistical analysis on these types of experiments, we must learn to account for the propagation of uncertainty from one step to the next. The propagation of uncertainty depends on the type of arithmetic operation performed on...
1.5K
Random Error01:04

Random Error

6.6K
Random or indeterminate errors originate from various uncontrollable variables, such as variations in environmental conditions, instrument imperfections, or the inherent variability of the phenomena being measured. Usually, these errors cannot be predicted, estimated, or characterized because their direction and magnitude often vary in magnitude and direction even during consecutive measurements. As a result, they are difficult to eliminate. However, the aggregate effect of these errors can be...
6.6K
Regression Toward the Mean01:52

Regression Toward the Mean

6.7K
Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when...
6.7K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Unraveling the geometry of visual relational reasoning.

Scientific reports·2026
Same author

Compositionality and systematicity emerge from iterated learning in deep linear networks.

Proceedings of the National Academy of Sciences of the United States of America·2026
Same author

Order parameters and phase transitions of continual learning in deep neural networks.

Proceedings of the National Academy of Sciences of the United States of America·2026
Same author

Interactions between long- and short-term synaptic plasticity transform temporal neural representations into spatial.

Proceedings of the National Academy of Sciences of the United States of America·2025
Same author

Coding schemes in neural networks learning classification tasks.

Nature communications·2025
Same author

Unraveling the Geometry of Visual Relational Reasoning.

ArXiv·2025

Related Experiment Video

Updated: Dec 6, 2025

Basics of Multivariate Analysis in Neuroimaging Data
06:35

Basics of Multivariate Analysis in Neuroimaging Data

Published on: July 24, 2010

17.2K

High-dimensional dynamics of generalization error in neural networks.

Madhu S Advani1, Andrew M Saxe1, Haim Sompolinsky2

  • 1Center for Brain Science, Harvard University, Cambridge, MA 02138, United States of America.

Neural Networks : the Official Journal of the International Neural Network Society
|October 6, 2020
PubMed
Summary

Large neural networks trained with gradient descent show natural protection against overfitting. Increasing network size in high-dimensional settings can reduce overtraining, contrary to some theories.

Keywords:
Generalization errorNeural networksRandom matrix theory

Related Experiment Videos

Last Updated: Dec 6, 2025

Basics of Multivariate Analysis in Neuroimaging Data
06:35

Basics of Multivariate Analysis in Neuroimaging Data

Published on: July 24, 2010

17.2K

Area of Science:

  • Machine Learning
  • Deep Learning Theory
  • Statistical Learning Theory

Background:

  • Understanding generalization dynamics in large neural networks is crucial for effective model training.
  • The high-dimensional regime, where parameters exceed data points, presents unique challenges for generalization.
  • Existing theories like Rademacher complexity may not fully capture deep neural network behavior.

Purpose of the Study:

  • To analyze the generalization dynamics of large neural networks trained with gradient descent.
  • To investigate the impact of data dimensionality and signal-to-noise ratio on learning.
  • To explore phenomena in overcomplete models and derive accurate generalization bounds.

Main Methods:

  • Analysis of average generalization dynamics using random matrix theory.
  • Exact solutions in linear models to derive error dynamics.
  • Investigation of non-linear neural networks in the high-dimensional regime.

Main Results:

  • Gradient descent naturally protects against overtraining and overfitting in large networks.
  • Overtraining is minimized at intermediate network sizes; larger networks can reduce overtraining.
  • High-dimensional regime benefits from small initial weights for low generalization error.

Conclusions:

  • Very large neural networks do not harm generalization and can reduce overtraining without regularization.
  • Novel phenomena include a 'frozen subspace' of weights and better-conditioned input correlations.
  • An alternative generalization bound is derived, accounting for these effects and matching simulations.