Search research articles

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

Related Concept Videos

The Uncertainty Principle

The Uncertainty Principle

Werner Heisenberg considered the limits of how accurately one can measure properties of an electron or other microscopic particles. He determined that there is a fundamental limit to how accurately one can measure both a particle’s position and its momentum simultaneously. The more accurate the measurement of the momentum of a particle is known, the less accurate the position at that time is known and vice versa. This is what is now called the Heisenberg uncertainty principle. He...

BIBO stability of continuous and discrete -time systems

BIBO stability of continuous and discrete -time systems

System stability is a fundamental concept in signal processing, often assessed using convolution. For a system to be considered bounded-input bounded-output (BIBO) stable, any bounded input signal must produce a bounded output signal. A bounded input signal is one where the modulus does not exceed a certain constant at any point in time.
To determine the BIBO stability, the convolution integral is utilized when a bounded continuous-time input is applied to a Linear Time-Invariant (LTI) system....

Entropy and the Second Law of Thermodynamics

Entropy and the Second Law of Thermodynamics

The second law of thermodynamics can be stated quantitatively using the concept of entropy. Entropy is the measure of disorder of the system.
The relation between entropy and disorder can be illustrated with the example of the phase change of ice to water. In ice, the molecules are located at specific sites giving a solid state, whereas, in a liquid form, these molecules are much freer to move. The molecular arrangement has therefore become more randomized. Although the change in average...

Propagation of Uncertainty from Random Error

Propagation of Uncertainty from Random Error

An experiment often consists of more than a single step. In this case, measurements at each step give rise to uncertainty. Because the measurements occur in successive steps, the uncertainty in one step necessarily contributes to that in the subsequent step. As we perform statistical analysis on these types of experiments, we must learn to account for the propagation of uncertainty from one step to the next. The propagation of uncertainty depends on the type of arithmetic operation performed on...

Propagation of Uncertainty from Systematic Error

Propagation of Uncertainty from Systematic Error

The atomic mass of an element varies due to the relative ratio of its isotopes. A sample's relative proportion of oxygen isotopes influences its average atomic mass. For instance, if we were to measure the atomic mass of oxygen from a sample, the mass would be a weighted average of the isotopic masses of oxygen in that sample. Since a single sample is not likely to perfectly reflect the true atomic mass of oxygen for all the molecules of oxygen on Earth, the mass we obtain from this...

Genetic Drift

Genetic Drift

Natural selection—probably the most well-known evolutionary mechanism—increases the prevalence of traits that enhance survival and reproduction. However, evolution does not merely propagate favorable traits, nor does it always benefit populations.

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same author

Data coarse graining can improve model performance.

Physical review. E·2026

Same author

Data coarse graining can improve model performance.

ArXiv·2025

Same author

Optimization and variability can coexist.

ArXiv·2025

Same author

Trait-space patterning is dictated by the tempo and mode of mutation.

Proceedings of the National Academy of Sciences of the United States of America·2025

Same author

Extreme Synergy in the Random-Energy Model.

Physical review letters·2025

Same author

Noise driven phase transitions in eco-evolutionary systems.

ArXiv·2023

Search research articles

Related Experiment Video

Updated: Aug 15, 2025

Quantification of Information Encoded by Gene Expression Levels During Lifespan Modulation Under Broad-range Dietary Restriction in C. elegans

Quantification of Information Encoded by Gene Expression Levels During Lifespan Modulation Under Broad-range Dietary Restriction in C. elegans

Published on: August 16, 2017

Perturbation Theory for the Information Bottleneck.

Vudtiwat Ngampruetikorn¹, David J Schwab¹

¹Initiative for the Theoretical Sciences, The Graduate Center, CUNY.

Advances in Neural Information Processing Systems

|January 4, 2023

Summary

We developed a perturbation theory for the information bottleneck (IB) method to analyze learning onset. Our findings offer the first complete characterization of maximum relevant information extraction from data.

Area of Science:

Machine Learning
Information Theory
Computational Neuroscience

Background:

Extracting relevant information is key for learning.
The Information Bottleneck (IB) method provides a formal framework for understanding learning.
The IB problem is computationally challenging due to its nonlinearity.

Purpose of the Study:

To derive a perturbation theory for the IB method.
To characterize the learning onset, defined as the limit of maximum relevant information per bit.
To provide a new perspective on the relationship between IB and the data processing inequality.

Main Methods:

Derivation of a perturbation theory for the IB method.
Testing the theory on synthetic probability distributions.

More Related Videos

Sealable Femtoliter Chamber Arrays for Cell-free Biology

Sealable Femtoliter Chamber Arrays for Cell-free Biology

Published on: March 11, 2015

Continuous Measurement of Biological Noise in Escherichia Coli Using Time-lapse Microscopy

Continuous Measurement of Biological Noise in Escherichia Coli Using Time-lapse Microscopy

Published on: April 27, 2021

Related Experiment Videos

Last Updated: Aug 15, 2025

Quantification of Information Encoded by Gene Expression Levels During Lifespan Modulation Under Broad-range Dietary Restriction in C. elegans

Quantification of Information Encoded by Gene Expression Levels During Lifespan Modulation Under Broad-range Dietary Restriction in C. elegans

Published on: August 16, 2017

Sealable Femtoliter Chamber Arrays for Cell-free Biology

Sealable Femtoliter Chamber Arrays for Cell-free Biology

Published on: March 11, 2015

Continuous Measurement of Biological Noise in Escherichia Coli Using Time-lapse Microscopy

Continuous Measurement of Biological Noise in Escherichia Coli Using Time-lapse Microscopy

Published on: April 27, 2021

Comparison with exact numerical solutions and previous theoretical attempts.

Main Results:

The first complete characterization of the learning onset.
Good agreement between the perturbation theory and numerical solutions near the learning onset.
Identification of a flawed assumption in previous perturbation theory attempts.

Conclusions:

The derived perturbation theory accurately describes the learning onset in the IB framework.
This work clarifies discrepancies with prior research by identifying a key assumption error.
The study deepens the understanding of the connection between information theory and learning processes.