Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Video

Updated: Oct 22, 2025

Quantification of Information Encoded by Gene Expression Levels During Lifespan Modulation Under Broad-range Dietary Restriction in C. elegans
09:23

Quantification of Information Encoded by Gene Expression Levels During Lifespan Modulation Under Broad-range Dietary Restriction in C. elegans

Published on: August 16, 2017

8.3K

Information Bottleneck Analysis by a Conditional Mutual Information Bound.

Taro Tezuka1, Shizuma Namekawa2

  • 1Faculty of Library, Information and Media Science, University of Tsukuba, Tsukuba, Ibaraki 305-8577, Japan.

Entropy (Basel, Switzerland)
|August 27, 2021
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

BIBO stability of continuous and discrete -time systems01:24

BIBO stability of continuous and discrete -time systems

603
System stability is a fundamental concept in signal processing, often assessed using convolution. For a system to be considered bounded-input bounded-output (BIBO) stable, any bounded input signal must produce a bounded output signal. A bounded input signal is one where the modulus does not exceed a certain constant at any point in time.
To determine the BIBO stability, the convolution integral is utilized when a bounded continuous-time input is applied to a Linear Time-Invariant (LTI) system....
603
Propagation of Uncertainty from Random Error00:59

Propagation of Uncertainty from Random Error

1.3K
An experiment often consists of more than a single step. In this case, measurements at each step give rise to uncertainty. Because the measurements occur in successive steps, the uncertainty in one step necessarily contributes to that in the subsequent step. As we perform statistical analysis on these types of experiments, we must learn to account for the propagation of uncertainty from one step to the next. The propagation of uncertainty depends on the type of arithmetic operation performed on...
1.3K
Difference from Background: Limit of Detection01:05

Difference from Background: Limit of Detection

7.4K
The limit of detection (LOD) is the smallest amount of analyte that can be distinguished from the background noise. The LOD value corresponds to the concentration at which the analyte signal is three times larger than the standard deviation of the blank signal. Below this value, the analyte signal cannot be differentiated from the background noise. It is calculated by dividing the calibration slope by 3 times the standard deviation of the blank signals.
The LOD indicates the presence or absence...
7.4K
Interpretation of Confidence Intervals01:19

Interpretation of Confidence Intervals

7.6K
A confidence interval is a better estimate of the population than a point estimate, as it uses a range of values from a sample instead of a single value.
Confidence intervals have confidence coefficients that are crucial for their interpretation. The most common confidence coefficients are 0.90, 0.95, and 0.99, which can be written as percentages–90%, 95%, and 99%, respectively.
Suppose a person calculates a confidence interval with a confidence coefficient of 0.95. In that case, they can...
7.6K
Central Limit Theorem01:14

Central Limit Theorem

17.6K
The central limit theorem, abbreviated as clt, is one of the most powerful and useful ideas in all of statistics. The central limit theorem for sample means says that if you repeatedly draw samples of a given size and calculate their means, and create a histogram of those means, then the resulting histogram will tend to have an approximate normal bell shape. In other words, as sample sizes increase, the distribution of means follows the normal distribution more closely.
The sample size, n, that...
17.6K
Propagation of Uncertainty from Systematic Error01:10

Propagation of Uncertainty from Systematic Error

1.0K
The atomic mass of an element varies due to the relative ratio of its isotopes. A sample's relative proportion of oxygen isotopes influences its average atomic mass. For instance, if we were to measure the atomic mass of oxygen from a sample, the mass would be a weighted average of the isotopic masses of oxygen in that sample. Since a single sample is not likely to perfectly reflect the true atomic mass of oxygen for all the molecules of oxygen on Earth, the mass we obtain from this...
1.0K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same journal

Research on a Regional Availability Evaluation Model for Road-Area High-Entropy Energy Based on Synergy Factors.

Entropy (Basel, Switzerland)·2026
Same journal

Atmospheric Turbulence Channel Modeling and Performance Analysis of a CO-ZP-OFDM Coherent Optical Communication System for UAV Air-to-Ground Scenarios.

Entropy (Basel, Switzerland)·2026
Same journal

Information Geometry and Asymptotic Theory for SMML Estimators.

Entropy (Basel, Switzerland)·2026
Same journal

Correlation Entropy and Power-Law Kinetics.

Entropy (Basel, Switzerland)·2026
Same journal

Research on the Contagion of Systemic Financial Risk Under the Impact of Climate Risks-From the Perspective of Complex Networks and Machine Learning.

Entropy (Basel, Switzerland)·2026
Same journal

The Statistical-Mechanical Meaning of the Wave Function of Quantum Mechanics.

Entropy (Basel, Switzerland)·2026
See all related articles

This study introduces conditional mutual information I(z;x|y) as a tighter bound for nuisance variables in supervised learning. This improves information bottleneck loss by better isolating relevant information for accurate predictions.

Area of Science:

  • Machine Learning
  • Information Theory

Background:

  • Supervised learning often uses information bottleneck loss to balance compression and prediction.
  • Task-nuisance decomposition aims to separate relevant information from irrelevant (nuisance) information.

Purpose of the Study:

  • To extend task-nuisance decomposition by introducing conditional mutual information I(z;x|y) as an alternative upper bound for nuisance variables.
  • To investigate the applicability of this bound even when latent representations are not sufficient.

Main Methods:

  • Utilized mutual information neural estimation (MINE) to estimate conditional mutual information I(z;x|y).
  • Compared I(z;x|y) with the traditional I(z;x) bound in experimental settings.

Main Results:

Keywords:
conditional mutual informationdeep learninginformation bottleneck

More Related Videos

A Machine Learning Approach to Design an Efficient Selective Screening of Mild Cognitive Impairment
12:18

A Machine Learning Approach to Design an Efficient Selective Screening of Mild Cognitive Impairment

Published on: January 11, 2020

7.7K
Using Three-color Single-molecule FRET to Study the Correlation of Protein Interactions
11:22

Using Three-color Single-molecule FRET to Study the Correlation of Protein Interactions

Published on: January 30, 2018

10.3K

Related Experiment Videos

Last Updated: Oct 22, 2025

Quantification of Information Encoded by Gene Expression Levels During Lifespan Modulation Under Broad-range Dietary Restriction in C. elegans
09:23

Quantification of Information Encoded by Gene Expression Levels During Lifespan Modulation Under Broad-range Dietary Restriction in C. elegans

Published on: August 16, 2017

8.3K
A Machine Learning Approach to Design an Efficient Selective Screening of Mild Cognitive Impairment
12:18

A Machine Learning Approach to Design an Efficient Selective Screening of Mild Cognitive Impairment

Published on: January 11, 2020

7.7K
Using Three-color Single-molecule FRET to Study the Correlation of Protein Interactions
11:22

Using Three-color Single-molecule FRET to Study the Correlation of Protein Interactions

Published on: January 30, 2018

10.3K
  • Demonstrated that I(z;x|y) serves as a tighter upper bound for nuisance information compared to I(z;x), especially in earlier network layers.
  • Observed differences in the information plane when using I(z;x|y) versus I(z;x), highlighting the impact of the tighter bound.

    • Conclusions:

    • Conditional mutual information I(z;x|y) offers a more effective objective for supervised learning by providing a tighter bound on nuisance variables.
    • This approach enhances the information bottleneck framework, leading to improved representation learning.