An Axiomatic Characterization of Mutual Information

Area of Science:

• Information Theory
• Probability Theory
• Mathematical Physics

Background:

• Shannon entropy is foundational for quantifying information.
• Existing characterizations of information measures often rely on specific mathematical formulations.
• Understanding the fundamental properties of mutual information is crucial for various fields.

Purpose of the Study:

• To provide a unique axiomatic characterization of mutual information.
• To introduce a novel axiom based on the concept of a Markov triangle.
• To develop coordinate-free proofs for information-theoretic concepts.

Main Methods:

• Axiomatic characterization of mutual information.
• Introduction of a new axiom involving Markov triangles.
• Coordinate-free mathematical derivations, avoiding logarithms.

Main Results:

• Mutual information is uniquely characterized by a set of axioms.
• A new axiom, based on Markov triangles, is introduced, with no analog for Shannon entropy.
• Proofs are presented in a coordinate-free manner, simplifying calculations.

Conclusions:

• The proposed axiomatic system offers a novel perspective on mutual information.
• The Markov triangle concept provides a new way to understand channel composition and conditional entropy.
• The coordinate-free approach facilitates a deeper, more general understanding of information measures.