Discovering Higher-Order Interactions Through Neural Information Decomposition

Area of Science:

• Information Theory
• Machine Learning
• Computational Neuroscience

Background:

• Complex data often exhibits higher-order functional dependencies among variables.
• Traditional models may misinterpret these complex patterns as noise due to a bias towards lower-order functions.
• Quantifying the contribution of different orders of dependence is crucial for accurate data analysis.

Purpose of the Study:

• To introduce a novel, theoretically grounded approach for information decomposition.
• To address the practical challenges of tractability and learnability in analyzing higher-order functions.
• To develop a method capable of distinguishing complex functional relationships from random noise in data.

Main Methods:

• Developed Neural Information Decomposition (NID), a new framework for information decomposition.
• Utilized neural networks for efficient estimation of information decomposition measures.
• Applied NID to synthetic datasets to evaluate its performance against conventional models.

Main Results:

• NID successfully learned to distinguish higher-order functions from noise in synthetic data.
• NID outperformed many unsupervised probability models in identifying complex data structures.
• The framework demonstrated practical utility in analyzing both biological and artificial neural networks.

Conclusions:

• Neural Information Decomposition (NID) provides an effective solution for analyzing complex, higher-order dependencies in data.
• NID overcomes practical limitations of existing information decomposition techniques, enabling efficient estimation.
• This approach offers a valuable tool for exploring intricate patterns in diverse neural network systems.