Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Encoding01:19

Encoding

331
Information enters the brain through encoding, which is the input of information into the memory system. Once sensory information is received from the environment, the brain labels or codes it. The information is then organized with similar information and connected to existing concepts. Encoding occurs through automatic processing and effortful processing.
Automatic processing involves the encoding of details like time, space, frequency, and the meaning of words, usually done without conscious...
331
Sequence Networks of Rotating Machines01:24

Sequence Networks of Rotating Machines

182
A Y-connected synchronous generator, grounded through a neutral impedance, is designed to produce balanced internal phase voltages with only positive-sequence components. The generator's sequence networks include a source voltage that is exclusively in the positive-sequence network. The sequence components of line-to-ground voltages at the generator terminals illustrate this configuration.
Zero-sequence current induces a voltage drop across the generator's neutral impedance and other...
182
Transformers in Distribution System01:27

Transformers in Distribution System

192
Transformers in distribution systems can be broadly categorized into distribution substation transformers and other distribution transformers. They are crucial for stepping down high transmission voltages to levels suitable for distribution and end-user applications.
Distribution substation transformers come in various ratings and typically use mineral oil for insulation and cooling. To prevent moisture and air from entering the oil, some transformers use an inert gas like nitrogen to fill the...
192
Multi-input and Multi-variable systems01:22

Multi-input and Multi-variable systems

207
Cruise control systems in cars are designed as multi-input systems to maintain a driver's desired speed while compensating for external disturbances such as changes in terrain. The block diagram for a cruise control system typically includes two main inputs: the desired speed set by the driver and any external disturbances, such as the incline of the road. By adjusting the engine throttle, the system maintains the vehicle's speed as close to the desired value as possible.
In the absence...
207
Signal Flow Graphs01:18

Signal Flow Graphs

381
Signal-flow graphs offer a streamlined and intuitive approach to representing control systems, providing an alternative to traditional block diagrams. These graphs use branches to symbolize systems and nodes to represent signals, effectively illustrating the relationships and interactions within the system.
In a signal-flow graph, branches denote the system's transfer functions, while nodes represent the signals. The direction of signal flow is indicated by arrows, with the corresponding...
381
Neural Circuits01:25

Neural Circuits

2.0K
Neural circuits and neuronal pools are two of the main structures found in the nervous system. Neural circuits are networks of neurons that work together to carry out a specific task or process. They consist of interconnected neurons and glial cells, which provide structural and metabolic support.
Neuronal pools are collections of nerve cells with similar functions and interact through chemical and electrical signals. These pools include both interneurons (the central neural circuit nodes that...
2.0K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Variational Garrote for Statistical Physics-based Sparse and Robust Variable Selection.

ArXiv·2026
Same author

Multiplicative Learning.

ArXiv·2025
Same author

Annealing approach to root finding.

Physical review. E·2024
Same author

Annealing approach to root-finding.

ArXiv·2024
Same author

Mirror Descent of Hopfield Model.

Neural computation·2023
Same author

Scale-invariant representation of machine learning.

Physical review. E·2022

Related Experiment Video

Updated: Oct 25, 2025

Author Spotlight: Advancing Alzheimer's Research – Exploring Early Detection and Multi-Omics Approaches
09:47

Author Spotlight: Advancing Alzheimer's Research – Exploring Early Detection and Multi-Omics Approaches

Published on: December 15, 2023

1.4K

Information Flows of Diverse Autoencoders.

Sungyeop Lee1, Junghyo Jo2,3

  • 1Department of Physics and Astronomy, Seoul National University, Seoul 08826, Korea.

Entropy (Basel, Switzerland)
|August 6, 2021
PubMed
Summary

Deep learning

Keywords:
autoencodersinformation bottleneck theorymatrix-based kernel estimationmutual information

More Related Videos

Decoding Natural Behavior from Neuroethological Embedding
08:00

Decoding Natural Behavior from Neuroethological Embedding

Published on: October 3, 2025

126

Related Experiment Videos

Last Updated: Oct 25, 2025

Author Spotlight: Advancing Alzheimer's Research – Exploring Early Detection and Multi-Omics Approaches
09:47

Author Spotlight: Advancing Alzheimer's Research – Exploring Early Detection and Multi-Omics Approaches

Published on: December 15, 2023

1.4K
Decoding Natural Behavior from Neuroethological Embedding
08:00

Decoding Natural Behavior from Neuroethological Embedding

Published on: October 3, 2025

126

Area of Science:

  • Artificial Intelligence
  • Information Theory
  • Machine Learning

Background:

  • Deep learning models excel in various applications.
  • Understanding their effectiveness is crucial.
  • Information theory offers insights into learning as data compression and transmission.

Purpose of the Study:

  • To investigate how network parameters influence information flow in deep learning.
  • To analyze information transmission and compression during the learning process.
  • To examine the role of a 'simplifying phase' in generalization.

Main Methods:

  • Utilized autoencoders (vanilla, sparse, tied, variational, label) as deep learning models.
  • Visualized information flow on an information plane using mutual information.
  • Quantified information flow with Rényi's matrix-based α-order entropy functional.

Main Results:

  • Autoencoders exhibit a fitting phase (increased mutual information) and sometimes a simplifying phase (decreased input-to-hidden mutual information).
  • Sparsity regularization amplifies the simplifying phase.
  • Tied, variational, and label autoencoders lack a simplifying phase.
  • All autoencoders achieved similar reconstruction errors, irrespective of the simplifying phase.

Conclusions:

  • The simplifying phase, where input-to-hidden information decreases, is not essential for generalization in autoencoders.
  • Network parameters like sparsity significantly shape information flow dynamics.
  • Information theory provides a valuable framework for analyzing deep learning mechanisms.