Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Entropy02:39

Entropy

Salt particles that have dissolved in water never spontaneously come back together in solution to reform solid particles. Moreover, a gas that has expanded in a vacuum remains dispersed and never spontaneously reassembles. The unidirectional nature of these phenomena is the result of a thermodynamic state function called entropy (S). Entropy is the measure of the extent to which the energy is dispersed throughout a system, or in other words, it is proportional to the degree of disorder of a...
Entropy01:18

Entropy

The first law of thermodynamics is quantitatively formulated via an equation relating the internal energy of a system, the heat exchanged by it, and the work done on it. A quantitative formulation of the second law of thermodynamics leads to defining a state function, the entropy.
When an ideal gas expands isothermally, the disorder in the gas increases. From the molecular perspective, the gas molecules have more volume to move around in.
Consider an infinitesimal step in the expansion, which...
The Second Law of Thermodynamics01:14

The Second Law of Thermodynamics

In the quest to identify a property that may reliably predict the spontaneity of a process, a promising candidate has been identified: entropy. Scientists refer to the measure of randomness or disorder within a system as entropy. High entropy means high disorder and low energy. To better understand entropy, think of a student’s bedroom. If no energy or work were put into it, the room would quickly become messy. It would exist in a very disordered state, one of high entropy. Energy must be put...
Entropy and the Second Law of Thermodynamics01:26

Entropy and the Second Law of Thermodynamics

Consider an isolated system in which a hot object is placed in contact with a cold one. This is an irreversible process that eventually leads both objects to reach the same equilibrium temperature. It is crucial to note that the constituents of any substance exhibit increased disorder at higher temperatures. As a cold substance absorbs heat, its constituents become more disordered. The energy transfer from a hotter object to a cooler one increases the system's disorder or randomness. This...
Entropy and the Second Law of Thermodynamics01:20

Entropy and the Second Law of Thermodynamics

The second law of thermodynamics can be stated quantitatively using the concept of entropy. Entropy is the measure of disorder of the system.
The relation  between entropy and disorder can be illustrated with the example of the phase change of ice to water. In ice, the molecules are located at specific sites giving a solid state, whereas, in a liquid form, these molecules are much freer to move. The molecular arrangement has therefore become more randomized. Although the change in average...
Second Law of Thermodynamics02:49

Second Law of Thermodynamics

In the quest to identify a property that may reliably predict the spontaneity of a process, a promising candidate has been identified: entropy. Processes that involve an increase in entropy of the system (ΔS > 0) are very often spontaneous; however, examples to the contrary are plentiful. By expanding consideration of entropy changes to include the surroundings, a significant conclusion regarding the relation between this property and spontaneity may be reached. In thermodynamic models, the...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Data-Driven Operational Bounds of Transmembrane Pressure for Modelling and Digital Twin Development in Haemodialysis and Haemodiafiltration.

Bioengineering (Basel, Switzerland)·2026
See all related articles

Related Experiment Video

Updated: Jun 27, 2026

Quantification of Information Encoded by Gene Expression Levels During Lifespan Modulation Under Broad-range Dietary Restriction in C. elegans
09:23

Quantification of Information Encoded by Gene Expression Levels During Lifespan Modulation Under Broad-range Dietary Restriction in C. elegans

Published on: August 16, 2017

Shannon Entropy and Beyond: An Information-Theoretic Framework for Randomness Pre-Screening.

Alexandru Dinu1

  • 1Faculty of Electronics, Telecommunications and Information Technology, National University of Science and Technology POLITEHNICA Bucharest, 061071 Bucharest, Romania.

Entropy (Basel, Switzerland)
|June 26, 2026
PubMed
Summary

A single entropy measure is insufficient for assessing data randomness. Combining Shannon, Rényi, permutation, and sample entropy provides a robust diagnostic profile for detecting pseudo-randomness in data sources.

Keywords:
PRNG validationRényi entropyShannon entropylottery datamachine learningpermutation entropysample entropy

More Related Videos

Applications of EEG Neuroimaging Data: Event-related Potentials, Spectral Power, and Multiscale Entropy
11:15

Applications of EEG Neuroimaging Data: Event-related Potentials, Spectral Power, and Multiscale Entropy

Published on: June 27, 2013

Related Experiment Videos

Last Updated: Jun 27, 2026

Quantification of Information Encoded by Gene Expression Levels During Lifespan Modulation Under Broad-range Dietary Restriction in C. elegans
09:23

Quantification of Information Encoded by Gene Expression Levels During Lifespan Modulation Under Broad-range Dietary Restriction in C. elegans

Published on: August 16, 2017

Applications of EEG Neuroimaging Data: Event-related Potentials, Spectral Power, and Multiscale Entropy
11:15

Applications of EEG Neuroimaging Data: Event-related Potentials, Spectral Power, and Multiscale Entropy

Published on: June 27, 2013

Area of Science:

  • Information Theory
  • Data Science
  • Statistical Analysis

Background:

  • Shannon entropy is commonly used to assess data randomness.
  • High Shannon entropy values are often misinterpreted as sufficient evidence of randomness.
  • Deterministic systems can exhibit high Shannon entropy, masking underlying predictability.

Purpose of the Study:

  • To demonstrate the limitations of using Shannon entropy alone for randomness assessment.
  • To introduce a multi-entropy diagnostic profile for more accurate randomness evaluation.
  • To validate the proposed method using real-world lottery data and pseudo-random number generators.

Main Methods:

  • Calculation of Shannon, Rényi, permutation, and sample entropy for various data sources.
  • Development of a combined entropy profile for comprehensive data analysis.
  • Application of a Random Forest classifier to distinguish between different data generation processes.

Main Results:

  • A deterministic logistic map shows high Shannon entropy but low permutation and sample entropy, indicating predictability.
  • The Romanian Loto 6/49 lottery data closely resembles a high-quality pseudo-random number generator (PRNG) across all four entropy measures.
  • The entropy deficit decay follows a power law, distinguishing predictable systems from random ones.

Conclusions:

  • A multi-entropy diagnostic profile is superior to single measures for identifying structured pseudo-randomness.
  • The method is effective for RNG certification, cryptographic auditing, and detecting non-random data.
  • The approach provides a domain-independent framework for rigorous randomness assessment.