Related Experiment Video
Updated: Sep 20, 2026

Quantification of Information Encoded by Gene Expression Levels During Lifespan Modulation Under Broad-range Dietary Restriction in C. elegans
Published on: August 16, 2017
Learning curves for mutual information maximization
1Institut für Theoretische Physik, Universität Würzburg, Am Hubland, D-97074 Würzburg, Germany.
Abstract:
An unsupervised learning procedure based on maximizing the mutual information between the outputs of two networks receiving different but statistically dependent inputs is analyzed [S. Becker and G. Hinton, Nature (London) 355, 161 (1992)]. For a generic data model, I show that in the large sample limit the structure in the data is recognized by mutual information maximization. For a more restricted model, where the networks are similar to perceptrons, I calculate the learning curves for zero-temperature Gibbs learning. These show that convergence can be rather slow, and a way of regularizing the procedure is considered.
Related Concept Videos
Maximizing the Directional Derivative
Methods of Medium Optimization
Introduction to Learning
In contrast to learned behaviors, unlearned behaviors such as crying, sexual...
Propagation of Uncertainty from Random Error
Observational Learning
Associative Learning
Classical conditioning, also known...