Probability Mass Exclusions and the Directed Components of Mutual Information

Area of Science:

• Information Theory
• Probability Theory
• Complex Systems Analysis

Background:

• Information is commonly understood as reducing uncertainty or restricting choices.
• A formal treatment of information in terms of exclusions is notably absent in existing literature.
• Understanding information distribution in complex systems is hindered by a lack of progress in this area.

Purpose of the Study:

• To formally characterize information using probability mass exclusions.
• To explore how different exclusions can result in the same information quantity.
• To gain insights into information sharing among random variables and directed information.

Main Methods:

• Developing an explicit characterization of information based on probability mass exclusions.
• Demonstrating the equivalence of different exclusions yielding the same information amount.
• Deriving a decomposition of mutual information to distinguish between exclusions.

Main Results:

• Information is formally characterized by probability mass exclusions.
• Multiple distinct exclusions can quantify the same amount of information.
• A novel decomposition of mutual information differentiates between exclusions, illuminating directed information.

Conclusions:

• The formalization of information via exclusions provides a new perspective on information theory.
• This approach offers critical insights into information sharing mechanisms within complex systems.
• The derived decomposition advances the understanding of directed information and its relationship to exclusions.