Introducing a differentiable measure of pointwise shared information.

Area of Science:

• Information Theory
• Machine Learning
• Statistical Inference

Background:

• Partial information decomposition (PID) quantifies information shared between variables.
• Existing PID measures lack differentiability, limiting their application.
• Previous approaches often incorporate external principles like decision theory.

Purpose of the Study:

• To develop a novel, differentiable measure for partial information decomposition.
• To derive this measure purely from information-theoretic principles.
• To explore its applications in areas like artificial neural networks.

Main Methods:

• Formulated a new measure based on local mutual information.
• Interpreted the measure through probability mass exclusions.
• Demonstrated its properties including Möbius inversion and target chain rule.

Main Results:

• The proposed measure is differentiable with respect to probability mass functions.
• It aligns with foundational information-theoretic concepts like Fano's mutual information.
• The measure is applicable to individual data points, facilitating local learning.

Conclusions:

• This differentiable PID measure advances information theory.
• It provides a principled way to understand information sharing.
• Its properties support applications in machine learning and agent decision-making.