Estimating the Unique Information of Continuous Variables

Area of Science:

• Computational Neuroscience
• Information Theory
• Machine Learning

Background:

• Neural systems integrate and transfer information, a fundamental process studied through partial information decomposition (PID).
• Existing PID methods are limited, particularly for general continuous distributions, leaving a gap in understanding complex information sharing.
• Investigating synergistic, redundant, and unique information contributions is key to deciphering neural computations.

Purpose of the Study:

• To develop a novel method for estimating unique information in continuous distributions for one-versus-two variable scenarios.
• To address the uncharted territory of PID for general continuous distributions.
• To apply the new method to brain-inspired neural models to reveal information processing mechanisms.

Main Methods:

• Developed a method combining copula decompositions and variational autoencoder optimization techniques.
• Solved the optimization problem over distributions with fixed bivariate marginals.
• Applied the method to analyze information flow in neural network models.

Main Results:

• Achieved excellent agreement with known analytic results for Gaussian distributions.
• Successfully recovered the effective connectivity of a chaotic rate neuron network.
• Uncovered intricate trade-offs between redundancy, synergy, and unique information in recurrent networks.

Conclusions:

• The new method provides a powerful tool for analyzing unique information in continuous distributions.
• Demonstrated the method's utility in understanding information processing in neural systems.
• Highlighted the complex interplay of information types in brain-inspired computational models.