Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Entropy02:39

Entropy

26.1K
Salt particles that have dissolved in water never spontaneously come back together in solution to reform solid particles. Moreover, a gas that has expanded in a vacuum remains dispersed and never spontaneously reassembles. The unidirectional nature of these phenomena is the result of a thermodynamic state function called entropy (S). Entropy is the measure of the extent to which the energy is dispersed throughout a system, or in other words, it is proportional to the degree of disorder of a...
26.1K
Entropy01:18

Entropy

2.8K
The first law of thermodynamics is quantitatively formulated via an equation relating the internal energy of a system, the heat exchanged by it, and the work done on it. A quantitative formulation of the second law of thermodynamics leads to defining a state function, the entropy.
When an ideal gas expands isothermally, the disorder in the gas increases. From the molecular perspective, the gas molecules have more volume to move around in.
Consider an infinitesimal step in the expansion, which...
2.8K
Entropy and the Second Law of Thermodynamics01:20

Entropy and the Second Law of Thermodynamics

3.3K
The second law of thermodynamics can be stated quantitatively using the concept of entropy. Entropy is the measure of disorder of the system.
The relation  between entropy and disorder can be illustrated with the example of the phase change of ice to water. In ice, the molecules are located at specific sites giving a solid state, whereas, in a liquid form, these molecules are much freer to move. The molecular arrangement has therefore become more randomized. Although the change in average...
3.3K
Propagation of Uncertainty from Random Error00:59

Propagation of Uncertainty from Random Error

1.9K
An experiment often consists of more than a single step. In this case, measurements at each step give rise to uncertainty. Because the measurements occur in successive steps, the uncertainty in one step necessarily contributes to that in the subsequent step. As we perform statistical analysis on these types of experiments, we must learn to account for the propagation of uncertainty from one step to the next. The propagation of uncertainty depends on the type of arithmetic operation performed on...
1.9K
Estimation of the Physical Quantities01:05

Estimation of the Physical Quantities

6.7K
On many occasions, physicists, other scientists, and engineers need to make estimates of a particular quantity. These are sometimes referred to as guesstimates, order-of-magnitude approximations, back-of-the-envelope calculations, or Fermi calculations. The physicist Enrico Fermi was famous for his ability to estimate various kinds of data with surprising precision. Estimating does not mean guessing a number or a formula at random. Instead, estimation means using prior experience and sound...
6.7K
The Entropy as a State Function01:14

The Entropy as a State Function

134
Consider an arbitrary process that moves between two specific states (A and B) in a cyclic manner. This process is reversible and broken down into smaller parts that each follow a Carnot cycle. A Carnot cycle has two isothermal (constant temperature) processes. During these processes, the ratio of the amount of heat transferred to their respective temperature remains constant. The other two processes in the Carnot cycle are also reversible but adiabatic, which means they occur without any heat...
134

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Vaginal microbiota transplantation: from microbial transfer to ecological restoration.

The Lancet. Microbe·2026
Same author

Discovery of Novel Dual Small-Molecule Inhibitors Targeting SHP2 and NAMPT for Overcoming Resistance to Allosteric SHP2 Inhibition.

Journal of medicinal chemistry·2026
Same author

A PbS quantum dot film as a hole transport layer for self-powered AgBiS<sub>2</sub> nanocrystal photodetectors.

Chemical communications (Cambridge, England)·2026
Same author

Association between serum methylmalonic acid and metabolic dysfunction-associated steatotic liver disease: NHANES-based study.

Medicine·2026
Same author

Targeted plasma proteomics reveals a central role of upregulated TNFRSF proteins in HIV-associated stroke.

Nature communications·2026
Same author

Transposon-based genome editing of industrial microorganisms: advances, challenges, and prospects.

Synthetic and systems biotechnology·2026
Same journal

A Model-Free Reinforcement Learning Implementation of Decision Making Under Uncertainty by Sequential Sampling.

Neural computation·2026
Same journal

DROP: Distributional and Regular Optimism and Pessimism for Reinforcement Learning.

Neural computation·2026
Same journal

Hierarchical Active Inference Using Successor Representations.

Neural computation·2026
Same journal

W-Kernel and Its Principal Space for Frequentist Evaluation of Bayesian Estimators.

Neural computation·2026
Same journal

A Hidden Markov Model-Inspired Sequence Classification Method for Hyperdimensional Computing.

Neural computation·2026
Same journal

Sparse Graphical Modeling for Electrophysiological Phase-Based Connectivity Using Circular Statistics.

Neural computation·2026
See all related articles

Related Experiment Video

Updated: May 3, 2026

One Dimensional Turing-Like Handshake Test for Motor Intelligence
14:05

One Dimensional Turing-Like Handshake Test for Motor Intelligence

Published on: December 15, 2010

24.8K

Entropy estimation in Turing's perspective.

Zhiyi Zhang1

  • 1Department of Mathematics and Statistics, University of North Carolina at Charlotte, Charlotte, NC 28223, USA. zzhang@uncc.edu

Neural Computation
|February 3, 2012
PubMed
Summary
This summary is machine-generated.

A novel nonparametric entropy estimator, based on Turing's formula, offers improved accuracy over the plug-in method. This new approach significantly enhances estimation precision for finite entropy distributions.

More Related Videos

Applications of EEG Neuroimaging Data: Event-related Potentials, Spectral Power, and Multiscale Entropy
11:15

Applications of EEG Neuroimaging Data: Event-related Potentials, Spectral Power, and Multiscale Entropy

Published on: June 27, 2013

34.9K
P300-Based Brain-Computer Interface Speller Performance Estimation with Classifier-Based Latency Estimation
06:09

P300-Based Brain-Computer Interface Speller Performance Estimation with Classifier-Based Latency Estimation

Published on: September 8, 2023

1.2K

Related Experiment Videos

Last Updated: May 3, 2026

One Dimensional Turing-Like Handshake Test for Motor Intelligence
14:05

One Dimensional Turing-Like Handshake Test for Motor Intelligence

Published on: December 15, 2010

24.8K
Applications of EEG Neuroimaging Data: Event-related Potentials, Spectral Power, and Multiscale Entropy
11:15

Applications of EEG Neuroimaging Data: Event-related Potentials, Spectral Power, and Multiscale Entropy

Published on: June 27, 2013

34.9K
P300-Based Brain-Computer Interface Speller Performance Estimation with Classifier-Based Latency Estimation
06:09

P300-Based Brain-Computer Interface Speller Performance Estimation with Classifier-Based Latency Estimation

Published on: September 8, 2023

1.2K

Area of Science:

  • Statistics
  • Information Theory
  • Nonparametric Statistics

Background:

  • Shannon's entropy is a fundamental measure in information theory.
  • The plug-in estimator is a common but often suboptimal method for entropy estimation.
  • Accurate entropy estimation is crucial for various data analysis tasks.

Purpose of the Study:

  • To propose and analyze a new nonparametric estimator for Shannon's entropy on countable alphabets.
  • To compare the proposed estimator's performance against the traditional plug-in estimator.
  • To establish theoretical guarantees for the new estimator's accuracy and convergence rates.

Main Methods:

  • Development of a new estimator based on Turing's formula.
  • Analysis of the estimator's properties using theoretical statistical methods.
  • Comparison of variance and mean squared error bounds with the plug-in estimator.

Main Results:

  • The proposed estimator demonstrates substantial gains in estimation accuracy.
  • A uniform variance upper bound of O(ln(n)/n) is established, outperforming the plug-in estimator's O([ln(n)]2/n).
  • For many subclasses, variance and mean squared error converge at a superior rate of O(1/n).
  • The estimator exhibits exponentially decaying bias for finite alphabets.

Conclusions:

  • The Turing's formula-based estimator offers a significant advancement in nonparametric entropy estimation.
  • The new estimator provides provably better accuracy and convergence rates compared to the plug-in method.
  • Further research includes developing bias-adjusted versions for even greater precision.