Related Experiment Video
Updated: Mar 29, 2026
11:15
Applications of EEG Neuroimaging Data: Event-related Potentials, Spectral Power, and Multiscale Entropy
Published on: June 27, 2013
34.5K
A Benchmark for Entropy Estimators
Lucio M Calcagnile1, Angelo Di Garbo1,2, Stefano Galatolo3
1Istituto di Biofisica, CNR, Via G. Moruzzi 1, 56124 Pisa, Italy.
Entropy (Basel, Switzerland)
|March 28, 2026
Summary
This study benchmarks entropy estimators for time series data. Approximate Entropy and symbolic methods accurately estimated Kolmogorov-Sinai entropy, unlike Sample and Permutation Entropy.
Area of Science:
- Dynamical Systems Theory
- Information Theory
- Time Series Analysis
Background:
- Estimating Kolmogorov-Sinai entropy is crucial for characterizing complex dynamical systems.
- Existing entropy estimators vary in accuracy and robustness across different data types and system dynamics.
- Computer-assisted proofs offer certified entropy values with rigorous error bounds for benchmarking.
Purpose of the Study:
- To quantitatively assess and compare the performance of widely used entropy estimators.
- To evaluate estimators on diverse one-dimensional dynamical systems with known certified entropy.
- To identify reliable entropy estimation techniques for numerical and symbolic time series data.
Main Methods:
- Generated long time series orbits for four classes of one-dimensional interval maps.
- Compared certified Kolmogorov-Sinai entropy values against estimates from Approximate Entropy, Sample Entropy, Permutation Entropy, symbolic Plug-In, and Non-Sequential Recursive Pair Substitution (NSRPS) methods.
- Utilized Grassberger-type bias correction for symbolic Plug-In and NSRPS estimators.
Main Results:
- Approximate Entropy and symbolic methods (Plug-In, NSRPS) demonstrated consistent accuracy within rigorous error bounds across all tested systems.
- Sample Entropy systematically underestimated the true entropy.
- Permutation Entropy exhibited significant biases, particularly for expanding maps lacking a Markov partition.
Conclusions:
- The study provides a quantitative benchmark for evaluating entropy estimation techniques in deterministic dynamical systems.
- Approximate Entropy and symbolic methods are recommended for reliable entropy estimation in similar systems.
- Further research is needed to improve the accuracy of Sample and Permutation Entropy estimators for complex dynamics.
More Related Videos
04:35Development of an Individual-Tree Basal Area Increment Model using a Linear Mixed-Effects Approach
Published on: July 3, 2020
3.8K
09:12Optimization of Processing of Tiebangchui with Highland Barley Wine Based on the Box-Behnken Design Combined with the Entropy Method
Published on: May 19, 2023
1.2K
Related Concept Videos
02:39Entropy
37.8K
Salt particles that have dissolved in water never spontaneously come back together in solution to reform solid particles. Moreover, a gas that has expanded in a vacuum remains dispersed and never spontaneously reassembles. The unidirectional nature of these phenomena is the result of a thermodynamic state function called entropy (S). Entropy is the measure of the extent to which the energy is dispersed throughout a system, or in other words, it is proportional to the degree of disorder of a...
37.8K
01:18Entropy
3.8K
The first law of thermodynamics is quantitatively formulated via an equation relating the internal energy of a system, the heat exchanged by it, and the work done on it. A quantitative formulation of the second law of thermodynamics leads to defining a state function, the entropy.
When an ideal gas expands isothermally, the disorder in the gas increases. From the molecular perspective, the gas molecules have more volume to move around in.
Consider an infinitesimal step in the expansion, which...
When an ideal gas expands isothermally, the disorder in the gas increases. From the molecular perspective, the gas molecules have more volume to move around in.
Consider an infinitesimal step in the expansion, which...
3.8K
01:26Estimating Population Standard Deviation
3.5K
When the population standard deviation is unknown and the sample size is large, the sample standard deviation s is commonly used as a point estimate of σ. However, it can sometimes under or overestimate the population standard deviation. To overcome this drawback, confidence intervals are determined to estimate population parameters and eliminate any calculation bias accurately. However, this only applies to random samples from normally distributed populations. Knowing the sample mean and...
3.5K
01:20Entropy and the Second Law of Thermodynamics
5.3K
The second law of thermodynamics can be stated quantitatively using the concept of entropy. Entropy is the measure of disorder of the system.
The relation between entropy and disorder can be illustrated with the example of the phase change of ice to water. In ice, the molecules are located at specific sites giving a solid state, whereas, in a liquid form, these molecules are much freer to move. The molecular arrangement has therefore become more randomized. Although the change in average...
The relation between entropy and disorder can be illustrated with the example of the phase change of ice to water. In ice, the molecules are located at specific sites giving a solid state, whereas, in a liquid form, these molecules are much freer to move. The molecular arrangement has therefore become more randomized. Although the change in average...
5.3K
01:26Entropy and the Second Law of Thermodynamics
262
Consider an isolated system in which a hot object is placed in contact with a cold one. This is an irreversible process that eventually leads both objects to reach the same equilibrium temperature. It is crucial to note that the constituents of any substance exhibit increased disorder at higher temperatures. As a cold substance absorbs heat, its constituents become more disordered. The energy transfer from a hotter object to a cooler one increases the system's disorder or randomness. This...
262
03:00Standard Entropy Change for a Reaction
25.7K
Entropy is a state function, so the standard entropy change for a chemical reaction (ΔS°rxn) can be calculated from the difference in standard entropy between the products and the reactants.
25.7K