Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Estimation of the Physical Quantities01:05

Estimation of the Physical Quantities

8.7K
On many occasions, physicists, other scientists, and engineers need to make estimates of a particular quantity. These are sometimes referred to as guesstimates, order-of-magnitude approximations, back-of-the-envelope calculations, or Fermi calculations. The physicist Enrico Fermi was famous for his ability to estimate various kinds of data with surprising precision. Estimating does not mean guessing a number or a formula at random. Instead, estimation means using prior experience and sound...
8.7K
What are Estimates?01:06

What are Estimates?

9.2K
It isn't easy to measure a parameter such as the mean height or the mean weight of a population. So, we draw samples from the population and calculate the mean height or mean weight of the individuals in the sample. This sample data acts as a representative measure of the population parameter. These sample statistics are known as estimates. 
The estimate for the mean of a sample is denoted by ͞x, whereas the mean of the population is designated as μ. Further, parameters such...
9.2K
Entropy02:39

Entropy

38.1K
Salt particles that have dissolved in water never spontaneously come back together in solution to reform solid particles. Moreover, a gas that has expanded in a vacuum remains dispersed and never spontaneously reassembles. The unidirectional nature of these phenomena is the result of a thermodynamic state function called entropy (S). Entropy is the measure of the extent to which the energy is dispersed throughout a system, or in other words, it is proportional to the degree of disorder of a...
38.1K
Entropy01:18

Entropy

3.8K
The first law of thermodynamics is quantitatively formulated via an equation relating the internal energy of a system, the heat exchanged by it, and the work done on it. A quantitative formulation of the second law of thermodynamics leads to defining a state function, the entropy.
When an ideal gas expands isothermally, the disorder in the gas increases. From the molecular perspective, the gas molecules have more volume to move around in.
Consider an infinitesimal step in the expansion, which...
3.8K
Estimating Population Standard Deviation01:26

Estimating Population Standard Deviation

3.5K
When the population standard deviation is unknown and the sample size is large, the sample standard deviation s is commonly used as a point estimate of σ. However, it can sometimes under or overestimate the population standard deviation. To overcome this drawback, confidence intervals are determined to estimate population parameters and eliminate any calculation bias accurately. However, this only applies to random samples from normally distributed populations. Knowing the sample mean and...
3.5K
Random Error01:04

Random Error

10.1K
Random or indeterminate errors originate from various uncontrollable variables, such as variations in environmental conditions, instrument imperfections, or the inherent variability of the phenomena being measured. Usually, these errors cannot be predicted, estimated, or characterized because their direction and magnitude often vary in magnitude and direction even during consecutive measurements. As a result, they are difficult to eliminate. However, the aggregate effect of these errors can be...
10.1K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Liquid chromatography/electrospray time-of-flight mass spectrometry for the characterisation of cyclic phosphazenes.

Rapid communications in mass spectrometry : RCM·2010
Same author

Entropy estimation of symbol sequences.

Chaos (Woodbury, N.Y.)·1996
Same author

Enlarged scaling ranges for the KS-entropy and the information dimension.

Chaos (Woodbury, N.Y.)·1996
See all related articles

Related Experiment Video

Updated: Apr 4, 2026

Applications of EEG Neuroimaging Data: Event-related Potentials, Spectral Power, and Multiscale Entropy
11:15

Applications of EEG Neuroimaging Data: Event-related Potentials, Spectral Power, and Multiscale Entropy

Published on: June 27, 2013

34.5K

A Note on Entropy Estimation.

Thomas Schürmann1

  • 1Jülich Supercomputing Centre, Jülich Research Centre, 52425 Jülich, Germany.

Neural Computation
|August 28, 2015
PubMed
Summary

We prove an entropy estimator H(z) is identical to H(1), with lower bias than plug-in methods. Numerical simulations show H(2) has less statistical error than H(z) for small samples.

Area of Science:

  • Information theory
  • Statistical estimation

Background:

  • Entropy estimation is crucial in various scientific fields.
  • Existing estimators like H(1), H(2), and H(z) have limitations in accuracy and bias.
  • Comparing these estimators is essential for reliable data analysis.

Purpose of the Study:

  • To compare the performance of entropy estimators H(z), H(1), and H(2).
  • To identify the identity H(z) ≡ H(1) and its implications.
  • To evaluate the bias and statistical error of these estimators, particularly in small sample regimes.

Main Methods:

  • Theoretical analysis to prove the identity H(z) ≡ H(1).
  • Bias analysis comparing H(1) with the ordinary likelihood (plug-in) estimator.
  • Numerical simulations to assess statistical error for H(2) versus H(z).

More Related Videos

Quantification of Information Encoded by Gene Expression Levels During Lifespan Modulation Under Broad-range Dietary Restriction in C. elegans
09:23

Quantification of Information Encoded by Gene Expression Levels During Lifespan Modulation Under Broad-range Dietary Restriction in C. elegans

Published on: August 16, 2017

8.7K
P300-Based Brain-Computer Interface Speller Performance Estimation with Classifier-Based Latency Estimation
06:09

P300-Based Brain-Computer Interface Speller Performance Estimation with Classifier-Based Latency Estimation

Published on: September 8, 2023

1.0K

Related Experiment Videos

Last Updated: Apr 4, 2026

Applications of EEG Neuroimaging Data: Event-related Potentials, Spectral Power, and Multiscale Entropy
11:15

Applications of EEG Neuroimaging Data: Event-related Potentials, Spectral Power, and Multiscale Entropy

Published on: June 27, 2013

34.5K
Quantification of Information Encoded by Gene Expression Levels During Lifespan Modulation Under Broad-range Dietary Restriction in C. elegans
09:23

Quantification of Information Encoded by Gene Expression Levels During Lifespan Modulation Under Broad-range Dietary Restriction in C. elegans

Published on: August 16, 2017

8.7K
P300-Based Brain-Computer Interface Speller Performance Estimation with Classifier-Based Latency Estimation
06:09

P300-Based Brain-Computer Interface Speller Performance Estimation with Classifier-Based Latency Estimation

Published on: September 8, 2023

1.0K

Main Results:

  • The identity H(z) ≡ H(1) is proven, correcting previous assumptions.
  • The bias of H(1) is shown to be less than or equal to the plug-in estimator.
  • H(2) demonstrates significantly smaller statistical error than H(z) in simulations for small samples and large event spaces.

Conclusions:

  • The identity H(z) ≡ H(1) simplifies entropy estimation comparisons.
  • H(1) offers an advantage in terms of bias reduction.
  • H(2) is a superior choice for statistical error minimization in challenging estimation scenarios.