Decomposing and Tracing Mutual Information by Quantifying Reachable Decision Regions

Area of Science:

• Information Theory
• Machine Learning
• Statistical Inference

Background:

• Partial Information Decomposition (PID) aims to attribute components of mutual information from multiple variables to a target.
• Existing PID measures face criticism for failing to satisfy axioms or yielding negative information components.
• There is a need for a robust PID measure that adheres to theoretical principles and provides non-negative decompositions.

Purpose of the Study:

• To develop a novel measure for partial information decomposition (PID).
• To ensure the new measure satisfies desired axioms, including an inclusion-exclusion principle.
• To provide a non-negative decomposition of information for an arbitrary number of variables.

Main Methods:

• Interpreting achievable Type I/II error pairs as pointwise uncertainty for target variable prediction.
• Constructing a distributive lattice with mutual information as a consistent valuation.
• Developing an algebra for the constructed measure based on reachable decision regions.

Main Results:

• The proposed PID measure satisfies the original axioms and an inclusion-exclusion principle.
• The decomposition yields non-negative components for any number of variables.
• Demonstrated practical applications in tracing information flow through Markov chains.

Conclusions:

• The developed PID measure offers a theoretically sound and practically applicable method for information decomposition.
• This approach can be utilized for analyzing information flow in complex systems like communication networks and data processing.
• The non-negative decomposition provides a more intuitive understanding of information sharing and synergy.