Search research articles

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

Related Concept Videos

Protein Networks

Protein Networks

An organism can have thousands of different proteins, and these proteins must cooperate to ensure the health of an organism. Proteins bind to other proteins and form complexes to carry out their functions. Many proteins interact with multiple other proteins creating a complex network of protein interactions.
These interactions can be represented through maps depicting protein-protein interaction networks, represented as nodes and edges. Nodes are circles that are representative of a protein,...

Protein Networks

Protein Networks

Network Covalent Solids

Network Covalent Solids

Network covalent solids contain a three-dimensional network of covalently bonded atoms as found in the crystal structures of nonmetals like diamond, graphite, silicon, and some covalent compounds, such as silicon dioxide (sand) and silicon carbide (carborundum, the abrasive on sandpaper). Many minerals have networks of covalent bonds.
To break or to melt a covalent network solid, covalent bonds must be broken. Because covalent bonds are relatively strong, covalent network solids are typically...

Neural Regulation

Neural Regulation

Digestion begins with a cephalic phase that prepares the digestive system to receive food. When our brain processes visual or olfactory information about food, it triggers impulses in the cranial nerves innervating the salivary glands and stomach to prepare for food.

Network Function of a Circuit

Network Function of a Circuit

Frequency response analysis in electrical circuits provides vital insights into a circuit's behavior as the frequency of the input signal changes. The transfer function, a mathematical tool, is instrumental in understanding this behavior. It defines the relationship between phasor output and input and comes in four types: voltage gain, current gain, transfer impedance, and transfer admittance. The critical components of the transfer function are the poles and zeros.

Neural Circuits

Neural Circuits

Neural circuits and neuronal pools are two of the main structures found in the nervous system. Neural circuits are networks of neurons that work together to carry out a specific task or process. They consist of interconnected neurons and glial cells, which provide structural and metabolic support.
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same author

Toward a Machine Learning Approach to Interpreting X-ray Spectra of Trace Impurities by Converting XANES to EXAFS.

The journal of physical chemistry. A·2024

Same author

Rethinking skip connections in Spiking Neural Networks with Time-To-First-Spike coding.

Frontiers in neuroscience·2024

Same author

Machine-learning-based spectral methods for partial differential equations.

Scientific reports·2023

Same author

Vibrational levels of a generalized Morse potential.

The Journal of chemical physics·2022

Same author

Optimal renormalization of multiscale systems.

Proceedings of the National Academy of Sciences of the United States of America·2021

Same author

Deep Learning to Generate <i>in Silico</i> Chemical Property Libraries and Candidate Molecules for Small Molecule Identification in Complex Samples.

Analytical chemistry·2019

Search research articles

Related Experiment Video

Updated: Feb 7, 2026

Author Spotlight: Enhancement of Salient Object Detection for Smart Grid Applications

Author Spotlight: Enhancement of Salient Object Detection for Smart Grid Applications

Published on: December 15, 2023

Doing the Impossible: Why Neural Networks Can Be Trained at All.

Nathan O Hodas¹, Panos Stinis¹

¹Pacific Northwest National Laboratory, Richland, WA, United States.

Frontiers in Psychology

|July 28, 2018

Summary

Deep neural networks can be trained effectively with limited data due to a mechanism that reduces the effective number of free parameters. This study uses mutual information to explain why structured networks achieve high performance, avoiding the curse of dimensionality.

Keywords:

correlation curse of dimensionality deep learning information theory mutual information neural networks training

More Related Videos

Deep Neural Networks for Image-Based Dietary Assessment

Deep Neural Networks for Image-Based Dietary Assessment

Published on: March 13, 2021

A Neural Network-Based Identification of Developmentally Competent or Incompetent Mouse Fully-Grown Oocytes

A Neural Network-Based Identification of Developmentally Competent or Incompetent Mouse Fully-Grown Oocytes

Published on: March 3, 2018

Related Experiment Videos

Last Updated: Feb 7, 2026

Author Spotlight: Enhancement of Salient Object Detection for Smart Grid Applications

Author Spotlight: Enhancement of Salient Object Detection for Smart Grid Applications

Published on: December 15, 2023

Deep Neural Networks for Image-Based Dietary Assessment

Deep Neural Networks for Image-Based Dietary Assessment

Published on: March 13, 2021

A Neural Network-Based Identification of Developmentally Competent or Incompetent Mouse Fully-Grown Oocytes

A Neural Network-Based Identification of Developmentally Competent or Incompetent Mouse Fully-Grown Oocytes

Published on: March 3, 2018

Area of Science:

Machine Learning
Computational Neuroscience
Statistical Physics

Background:

Deep neural networks (DNNs) face training challenges as their size increases, raising questions about data requirements.
Similar challenges exist in protein folding, spin glasses, and biological neural networks, where complex systems find optimal configurations efficiently.

Purpose of the Study:

To investigate the mechanism enabling reliable training of DNNs with limited data.
To elucidate how complex systems find optimal configurations.
To propose methods for accelerating DNN training by exploiting this mechanism.

Main Methods:

Utilizing the concept of mutual information between successive layers in DNNs.
Analyzing the impact of network structure on mutual information.
Relating high mutual information to a reduced effective number of free parameters.

Main Results:

Adding structure to DNNs increases mutual information between layers.
High mutual information implies an exponentially smaller effective parameter count compared to tunable weights.
This finding provides insight into why DNNs with more weights than training samples can be reliably trained.

Conclusions:

A mechanism exists that forces complex systems, including DNNs, into low-dimensional manifolds, mitigating the curse of dimensionality.
Mutual information serves as a key metric to understand and potentially exploit this mechanism.
Structured deep learning models exhibit higher mutual information, leading to more efficient and reliable training.