Exact Computation of Transfer Entropy with Path Weight Sampling

Area of Science:

• Information Theory
• Computational Neuroscience
• Complex Systems

Background:

• Quantifying directional information flow is crucial for understanding natural and engineered systems.
• Transfer entropy is a key measure for information flow but often relies on uncontrolled approximations.
• Existing methods struggle with complex models, including those with hidden variables, nonlinearity, and feedback.

Purpose of the Study:

• Introduce a novel computational algorithm, transfer entropy-path weight sampling (TE-PWS), for exact transfer entropy quantification.
• Enable precise measurement of information flow in diverse stochastic models.
• Address limitations of approximate methods in complex dynamical systems.

Main Methods:

• Developed TE-PWS, a computational algorithm leveraging polymer and path sampling techniques.
• Computed transfer entropy as a Monte Carlo average over signal trajectory space.
• Applied TE-PWS to linear and nonlinear systems, including those with feedback.

Main Results:

• TE-PWS allows exact quantification of transfer entropy for any stochastic model.
• Demonstrated that common approximate methods yield significant systematic errors and high computational costs.
• Showcased TE-PWS's ability to handle complex scenarios like multiple hidden variables, nonlinearity, and feedback.

Conclusions:

• TE-PWS provides an exact and efficient method for calculating transfer entropy.
• The study highlights the inaccuracies of previously used approximate methods.
• TE-PWS offers a powerful tool for analyzing information flow in complex systems, overcoming challenges posed by feedback loops.