Explicit mutual information for simple networks and neurons with lognormal activities

Area of Science:

• Computational neuroscience
• Information theory
• Statistical physics

Background:

• Many natural networks exhibit heavy-tailed lognormal distributions.
• Understanding information flow in these complex systems is crucial.

Purpose of the Study:

• To derive exact information-theoretic characterizations for networks with lognormally distributed stochastic variables.
• To analyze mutual information in neural networks with lognormally distributed activities and synaptic weights.

Main Methods:

• Derivation of analytical formulas for mutual information.
• Comparison between heavy-tailed (lognormal) and short-tailed distributions.

Main Results:

• Analytical formulas for mutual information between network elements were derived.
• Mutual information is generally larger in lognormal neural networks compared to short-tailed ones.
• Mutual information can diverge for finite variances in lognormal neural networks.

Conclusions:

• Heavy-tailed lognormal distributions in neural activity and synaptic weights enhance mutual information.
• Evolution may favor heterogeneous neural dynamics for optimized information processing.