Related Experiment Video
Updated: Oct 22, 2025

Applications of EEG Neuroimaging Data: Event-related Potentials, Spectral Power, and Multiscale Entropy
Published on: June 27, 2013
A Review of Shannon and Differential Entropy Rate Estimation
Andrew Feutrill1,2,3, Matthew Roughan2,3
1CSIRO/Data61, 13 Kintore Avenue, Adelaide, SA 5000, Australia.
Abstract:
In this paper, we present a review of Shannon and differential entropy rate estimation techniques. Entropy rate, which measures the average information gain from a stochastic process, is a measure of uncertainty and complexity of a stochastic process. We discuss the estimation of entropy rate from empirical data, and review both parametric and non-parametric techniques. We look at many different assumptions on properties of the processes for parametric processes, in particular focussing on Markov and Gaussian assumptions. Non-parametric estimation relies on limit theorems which involve the entropy rate from observations, and to discuss these, we introduce some theory and the practical implementations of estimators of this type.
More Related Videos
09:23Quantification of Information Encoded by Gene Expression Levels During Lifespan Modulation Under Broad-range Dietary Restriction in C. elegans
Published on: August 16, 2017
10:10Three Differential Expression Analysis Methods for RNA Sequencing: limma, EdgeR, DESeq2
Published on: September 18, 2021
Related Concept Videos
Entropy and the Second Law of Thermodynamics
The relation between entropy and disorder can be illustrated with the example of the phase change of ice to water. In ice, the molecules are located at specific sites giving a solid state, whereas, in a liquid form, these molecules are much freer to move. The molecular arrangement has therefore become more randomized. Although the change in average...
Entropy
The Second Law of Thermodynamics
Standard Entropy Change for a Reaction
Entropy Change in Reversible Processes
The statement can be further generalized to prove that entropy is a state function. Take a cyclic process between any two points on a p-V diagram.
Second Law of Thermodynamics