Nonadditive entropies yield probability distributions with biases not warranted by the data

Area of Science:

• Information Theory
• Statistical Mechanics
• Probability Theory

Background:

• Variational principles commonly use entropy to infer probability distributions from limited data.
• The Boltzmann-Gibbs entropy, when maximized, ensures inferred distributions satisfy probability rules for independent events, as per Shore and Johnson's axioms.
• Certain nonadditive entropies, such as Tsallis entropy, do not adhere to these axioms.

Purpose of the Study:

• To investigate the axiomatic consistency of various entropy measures in probability distribution inference.
• To demonstrate how nonadditive entropy functions can introduce biases not supported by data.
• To apply the Shore and Johnson axiomatic framework to analyze deviations caused by nonadditive entropies.

Main Methods:

• Utilizing the axiomatic framework established by Shore and Johnson.
• Analyzing the properties of probability distributions inferred using different entropy measures.
• Comparing the consistency of Boltzmann-Gibbs entropy with nonadditive entropies like Tsallis entropy.

Main Results:

• Nonadditive entropies, including Tsallis entropy, violate the Shore and Johnson axioms for probability distributions.
• This violation leads to the generation of biases in inferred probability distributions.
• These biases are not justified by the underlying data, indicating a fundamental inconsistency.

Conclusions:

• The Boltzmann-Gibbs entropy is essential for maintaining the multiplicative consistency of probability distributions derived from limited data.
• Nonadditive entropy functions, such as Tsallis entropy, are axiomatically inconsistent and introduce unwarranted biases.
• Adherence to the Shore and Johnson axioms is crucial for reliable probability distribution inference, particularly when dealing with limited or incomplete data.