Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Gradient and Del Operator01:14

Gradient and Del Operator

4.2K
In mathematics and physics, the gradient and del operator are fundamental concepts used to describe the behavior of functions and fields in space. The gradient is a mathematical operator that gives both the magnitude and direction of the maximum spatial rate of change. Consider a person standing on a mountain. The slope of the mountain at any given point is not defined unless it is quantified in a particular direction. For this reason, a "directional derivative" is defined, which is a vector...
4.2K
Hierarchy of Motor Control01:18

Hierarchy of Motor Control

5.7K
The hierarchy of motor control refers to the different levels of organization and processing involved in controlling movement in the body. These levels range from higher cortical areas involved in planning and decision-making to lower spinal cord reflexes that respond automatically to external stimuli.
5.7K
Control Systems01:10

Control Systems

1.7K
Control systems are everywhere in contemporary society, influencing diverse applications from aerospace to automated manufacturing. These systems can be found naturally within biological processes, such as blood sugar regulation and heart rate adjustment in response to stress, as well as in man-made systems like elevators and automated vehicles. A control system is essentially a network of subsystems and processes that collaboratively convert specific inputs into desired outputs.
At the heart...
1.7K
Multi-input and Multi-variable systems01:22

Multi-input and Multi-variable systems

334
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence of...
334
Neural Regulation01:37

Neural Regulation

43.0K
Digestion begins with a cephalic phase that prepares the digestive system to receive food. When our brain processes visual or olfactory information about food, it triggers impulses in the cranial nerves innervating the salivary glands and stomach to prepare for food.
43.0K
Neural Control of Respiration01:18

Neural Control of Respiration

4.3K
The neural regulation of respiration is a meticulously coordinated process primarily controlled by the respiratory centers located within the brainstem. These centers, composed of specialized neurons, transmit nerve impulses that control the contraction and relaxation of our respiratory muscles.
Respiratory Centers in the Brainstem
Two primary areas comprise the respiratory center: the medullary respiratory center in the medulla oblongata and the pontine respiratory group in the pons. The...
4.3K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

What if eye...? Computationally recreating vision evolution.

Science advances·2025
Same author

Dynamics in Deep Classifiers Trained with the Square Loss: Normalization, Low Rank, Neural Collapse, and Generalization Bounds.

Research (Washington, D.C.)·2023
Same author

Theoretical issues in deep networks.

Proceedings of the National Academy of Sciences of the United States of America·2020
Same author

Scale and translation-invariance for novel objects in human vision.

Scientific reports·2020
Same author

Invariant recognition drives neural representations of action sequences.

PLoS computational biology·2017
Same author

A fast, invariant representation for human action in the visual system.

Journal of neurophysiology·2017
Same journal

Chlorinated VSLSs Surpass HCFCs in CFC-11-Equivalent Emissions for Ozone Layer Depletion in China.

Nature communications·2026
Same journal

Author Correction: Charge transfer in triphenylamine-tetrazine covalent organic frameworks for solar-driven hydrogen peroxide production.

Nature communications·2026
Same journal

Vegetation browning patterns under compound soil and atmospheric dryness in northern permafrost ecosystems.

Nature communications·2026
Same journal

Voltage imaging of CA1 pyramidal cells and SST+ interneurons reveals stability and plasticity mechanisms of spatial firing.

Nature communications·2026
Same journal

Radical-omics reveals the hydrogen-abstraction pathway of isoprene oxidation.

Nature communications·2026
Same journal

Toughening elastomer via sequentially activated multi-pathway energy dissipation.

Nature communications·2026
See all related articles

Related Experiment Video

Updated: Dec 27, 2025

Deep Neural Networks for Image-Based Dietary Assessment
13:19

Deep Neural Networks for Image-Based Dietary Assessment

Published on: March 13, 2021

9.9K

Complexity control by gradient descent in deep networks.

Tomaso Poggio1, Qianli Liao2, Andrzej Banburski2

  • 1Center for Brains, Minds, and Machines, MIT, Cambridge, Massachusetts, USA. tp@ai.mit.edu.

Nature Communications
|February 26, 2020
PubMed
Summary
This summary is machine-generated.

Overparametrized deep networks achieve good predictions without explicit complexity control. Gradient descent provides effective regularization for exponential loss functions by normalizing weights crucial for classification.

More Related Videos

Author Spotlight: Enhancement of Salient Object Detection for Smart Grid Applications
03:31

Author Spotlight: Enhancement of Salient Object Detection for Smart Grid Applications

Published on: December 15, 2023

938

Related Experiment Videos

Last Updated: Dec 27, 2025

Deep Neural Networks for Image-Based Dietary Assessment
13:19

Deep Neural Networks for Image-Based Dietary Assessment

Published on: March 13, 2021

9.9K
Author Spotlight: Enhancement of Salient Object Detection for Smart Grid Applications
03:31

Author Spotlight: Enhancement of Salient Object Detection for Smart Grid Applications

Published on: December 15, 2023

938

Area of Science:

  • Machine Learning
  • Deep Learning Theory
  • Optimization

Background:

  • Overparametrized deep networks often generalize well despite lacking explicit regularization.
  • Understanding the mechanisms behind this phenomenon is crucial for developing more robust models.

Purpose of the Study:

  • To explain the effective regularization in overparametrized deep networks.
  • To identify the role of gradient descent in controlling network complexity.

Main Methods:

  • Analysis of gradient descent dynamics for exponential-type loss functions.
  • Investigation of normalized weights relevant for classification tasks.

Main Results:

  • Gradient descent exhibits an implicit regularization effect.
  • This regularization is observed in the normalized weights of the network.
  • The effect is particularly pronounced for exponential-type loss functions.

Conclusions:

  • The study resolves the puzzle of good prediction in overparametrized networks.
  • Gradient descent acts as a natural regularizer, controlling complexity through weight normalization.
  • Findings offer insights into the theoretical underpinnings of deep learning generalization.