Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Videos

Predictability, complexity, and learning.

W Bialek1, I Nemenman, N Tishby

  • 1NEC Research Institute, Princeton, NJ 08540, USA.

Neural Computation
|October 25, 2001
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Effect of butyrate sources in a high-concentrate diet on rumen structure and function in growing rams.

Animal : an international journal of animal bioscience·2024
Same author

Quantum and classical dynamics in biochemical reactions.

Photosynthesis research·2014
Same author

The Third q-bio Conference on Cellular Information Processing.

IET systems biology·2010
Same author

Time-dependent corrections to effective rate and event statistics in Michaelis-Menten kinetics.

IET systems biology·2010
Same author

Multivariate dependence and genetic networks inference.

IET systems biology·2010
Same author

Quantifying evolvability in small biological networks.

IET systems biology·2010
Same journal

A Model-Free Reinforcement Learning Implementation of Decision Making Under Uncertainty by Sequential Sampling.

Neural computation·2026
Same journal

DROP: Distributional and Regular Optimism and Pessimism for Reinforcement Learning.

Neural computation·2026
Same journal

Hierarchical Active Inference Using Successor Representations.

Neural computation·2026
Same journal

W-Kernel and Its Principal Space for Frequentist Evaluation of Bayesian Estimators.

Neural computation·2026
Same journal

A Hidden Markov Model-Inspired Sequence Classification Method for Hyperdimensional Computing.

Neural computation·2026
Same journal

Sparse Graphical Modeling for Electrophysiological Phase-Based Connectivity Using Circular Statistics.

Neural computation·2026
See all related articles

We introduce predictive information, a measure of time series complexity. It can grow logarithmically or as a power law, offering insights into model complexity and system dynamics.

Area of Science:

  • Information theory
  • Statistical mechanics
  • Dynamical systems theory

Background:

  • Predictive information quantifies the mutual information between a time series' past and future.
  • Understanding time series complexity is crucial in physics, statistics, and biology.

Purpose of the Study:

  • To define and analyze predictive information for time series.
  • To explore the relationship between predictive information, model complexity, and system dynamics.

Main Methods:

  • Mathematical definition of predictive information I(pred)(T).
  • Analysis of I(pred)(T) in the limit of large observation times (T).
  • Comparison with complexity measures from learning theory and statistical mechanics.

Main Results:

Related Experiment Videos

  • Identified three behaviors for I(pred)(T): finite, logarithmic growth, or fractional power-law growth.
  • Logarithmic growth correlates with finite-parameter models; power-law growth with infinite-parameter models.
  • The divergent part of I(pred)(T) serves as a unique measure of underlying dynamic complexity.

Conclusions:

  • Predictive information offers a unified framework for understanding time series complexity.
  • This measure connects information theory with concepts from learning theory and statistical physics.
  • Potential applications span diverse fields including physics, statistics, and biology.