Related Experiment Video
Updated: Jun 3, 2025

Applications of EEG Neuroimaging Data: Event-related Potentials, Spectral Power, and Multiscale Entropy
Published on: June 27, 2013
An Intrinsic Characterization of Shannon's and Rényi's Entropy
Martin Schlather1, Carmen Ditscheid1
1Institute of Mathematics, University of Mannheim, 68131 Mannheim, Germany.
Abstract:
All characterizations of the Shannon entropy include the so-called chain rule, a formula on a hierarchically structured probability distribution, which is based on at least two elementary distributions. We show that the chain rule can be split into two natural components, the well-known additivity of the entropy in case of cross-products and a variant of the chain rule that involves only a single elementary distribution. The latter is given as a proportionality relation and, hence, allows a vague interpretation as self-similarity, hence intrinsic property of the Shannon entropy. Analogous characterizations are given for the Rényi entropy and its limits, the min-entropy and the Hartley entropy.
More Related Videos
Related Concept Videos
Entropy and the Second Law of Thermodynamics
The relation between entropy and disorder can be illustrated with the example of the phase change of ice to water. In ice, the molecules are located at specific sites giving a solid state, whereas, in a liquid form, these molecules are much freer to move. The molecular arrangement has therefore become more randomized. Although the change in average...
Entropy
Entropy Change in Reversible Processes
The statement can be further generalized to prove that entropy is a state function. Take a cyclic process between any two points on a p-V diagram.
The Second Law of Thermodynamics
Second Law of Thermodynamics
Third Law of Thermodynamics

