Related Experiment Video
Updated: May 14, 2026
11:15
Applications of EEG Neuroimaging Data: Event-related Potentials, Spectral Power, and Multiscale Entropy
Published on: June 27, 2013
Expanding the transfer entropy to identify information subgraphs in complex systems
S Stramaglia1, Guo-Rong Wu, M Pellicoro
1Dipartimento di Fisica, Universitá degli Studi di Bari and Istituto Nazionale di Fisica Nucleare, Sezione di Bari, via Orabona 4, 70126 Bari, Italy.
Summary
We expanded transfer entropy to identify key variables predicting future states. High values reveal information circuits, indicating synergistic or redundant information flow in complex systems.
Area of Science:
- Information theory
- Complexity science
- Neuroscience
Background:
- Transfer entropy quantifies information flow between variables.
- Identifying causal relationships and information processing in complex systems is challenging.
- Existing methods may not fully capture irreducible information contributions.
Purpose of the Study:
- To formally expand transfer entropy for identifying irreducible sets of predictive variables.
- To characterize informational circuits within systems based on variable contributions.
- To investigate the synergetic or redundant nature of information transfer.
Main Methods:
- Formal mathematical expansion of transfer entropy.
- Development of a method to identify "multiplets" of variables.
- Analysis of variable contributions and their informational character (synergetic/redundant).
Main Results:
- The proposed method identifies irreducible sets of variables crucial for predicting future states.
- High "multiplet" values indicate the presence of informational circuits.
- The sign of contribution reveals whether information flow is synergetic or redundant.
Conclusions:
- The expanded transfer entropy offers a novel approach to dissecting information processing in complex systems.
- This framework can reveal underlying informational architectures and their dynamics.
- Preliminary results show promise in analyzing neuroimaging data (fMRI, EEG).
Related Concept Videos
01:10Entropy Change in Reversible Processes
In the Carnot engine, which achieves the maximum efficiency between two reservoirs of fixed temperatures, the total change in entropy is zero. The observation can be generalized by considering any reversible cyclic process consisting of many Carnot cycles. Thus, it can be stated that the total entropy change of any ideal reversible cycle is zero.
The statement can be further generalized to prove that entropy is a state function. Take a cyclic process between any two points on a p-V diagram.
The statement can be further generalized to prove that entropy is a state function. Take a cyclic process between any two points on a p-V diagram.
01:21Entropy Changes Accompanying Specific Processes
Entropy, a measure of disorder in a system, changes during phase transitions like freezing or boiling. At the transition temperature Ttrs, where two phases are in equilibrium, the phase transition is a reversible process. The entropy change can be calculated from a substance's enthalpy of transition using the equation ΔStrs = ΔtrsH /Ttrs.When a perfect gas expands isothermally from one volume to another, entropy increases logarithmically with volume. Conversely, isothermal compression results...
02:39Entropy
Salt particles that have dissolved in water never spontaneously come back together in solution to reform solid particles. Moreover, a gas that has expanded in a vacuum remains dispersed and never spontaneously reassembles. The unidirectional nature of these phenomena is the result of a thermodynamic state function called entropy (S). Entropy is the measure of the extent to which the energy is dispersed throughout a system, or in other words, it is proportional to the degree of disorder of a...
01:18Entropy
The first law of thermodynamics is quantitatively formulated via an equation relating the internal energy of a system, the heat exchanged by it, and the work done on it. A quantitative formulation of the second law of thermodynamics leads to defining a state function, the entropy.
When an ideal gas expands isothermally, the disorder in the gas increases. From the molecular perspective, the gas molecules have more volume to move around in.
Consider an infinitesimal step in the expansion, which...
When an ideal gas expands isothermally, the disorder in the gas increases. From the molecular perspective, the gas molecules have more volume to move around in.
Consider an infinitesimal step in the expansion, which...
01:27State Space Representation
The frequency-domain technique, commonly used in analyzing and designing feedback control systems, is effective for linear, time-invariant systems. However, it falls short when dealing with nonlinear, time-varying, and multiple-input multiple-output systems. The time-domain or state-space approach addresses these limitations by utilizing state variables to construct simultaneous, first-order differential equations, known as state equations, for an nth-order system.
Consider an RLC circuit, a...
Consider an RLC circuit, a...
01:14The Entropy as a State Function
Consider an arbitrary process that moves between two specific states (A and B) in a cyclic manner. This process is reversible and broken down into smaller parts that each follow a Carnot cycle. A Carnot cycle has two isothermal (constant temperature) processes. During these processes, the ratio of the amount of heat transferred to their respective temperature remains constant. The other two processes in the Carnot cycle are also reversible but adiabatic, which means they occur without any heat...