Non-Negative Decomposition of Multivariate Information: From Minimum to Blackwell-Specific Information

Area of Science:

• Information Theory
• Statistical Inference
• Machine Learning

Background:

• Partial Information Decomposition (PID) analyzes information sharing from source to target variables.
• Existing PID methods face challenges with arbitrary discrete variables and suitable decomposition measures.
• Lattice-based approaches have gained traction but require further development for general applicability.

Purpose of the Study:

• To propose a novel, non-negative partial information decomposition (PID) for arbitrary discrete random variables.
• To develop a PID measure that satisfies an inclusion-exclusion relation for any f-information measure.
• To provide a robust framework for analyzing information flow and redundancy.

Main Methods:

• Constructing a pointwise PID based on the target variable's perspective.
• Utilizing the equivalence between Blackwell and zonogon order for discrete variables.
• Defining f-information as the expected value of quantifying the boundary of Neyman-Pearson regions (zonogons).

Main Results:

• A new non-negative PID is proposed, satisfying inclusion-exclusion for any f-information measure.
• The decomposition is proven to adhere to desired axioms and yield non-negative partial information.
• Demonstration of transformability between different decomposition lattices and non-negative Rényi-information decomposition.

Conclusions:

• The developed PID offers a mathematically sound and practically applicable method for information decomposition.
• The approach guarantees non-negative results and flexibility across different information measures.
• The decomposition facilitates tracing information flow through Markov chains and understanding complex variable interactions.