Related Experiment Video
Updated: Jul 7, 2025
11:15
Applications of EEG Neuroimaging Data: Event-related Potentials, Spectral Power, and Multiscale Entropy
Published on: June 27, 2013
33.7K
Boltzmann-Shannon interaction entropy: A normalized measure for continuous variables with an application as a
C Tyler Diggans1, Abd AlRahman R AlMomani2
1Air Force Research Laboratory Information Directorate, 525 Brooks Rd., Rome, New York 13441, USA.
Chaos (Woodbury, N.Y.)
|December 27, 2023
Summary
A new Boltzmann-Shannon interaction entropy (BSIE) offers a sample-size-independent measure of uncertainty. This stable, normalized entropy is computationally efficient and useful for assessing data quality in regression.
Area of Science:
- Information Theory
- Statistical Modeling
Background:
- Geometric partition entropy offers an alternative to differential Shannon entropy for quantifying uncertainty.
- It improves upon traditional estimators by handling sparse samples and extreme outliers.
Purpose of the Study:
- To define a new, normalized entropy measure unbiased by sample size.
- To introduce the Boltzmann-Shannon interaction entropy (BSIE) for improved uncertainty quantification.
Main Methods:
- Leveraging a complementary relationship between geometric and frequency-based entropy approaches.
- Defining BSIE using a standard divergence between measure-based and frequency-based distributions.
- Estimating BSIE in a computationally efficient, parameter-free manner.
Main Results:
- Developed a stable, normalized entropy measure (BSIE) independent of sample size.
- Demonstrated BSIE's utility as a quality metric for subsampling.
- Applied BSIE in the context of nonlinear polynomial regression.
Conclusions:
- BSIE provides a robust and efficient method for uncertainty quantification.
- The new entropy measure enhances data analysis, particularly in regression tasks.
- BSIE represents a significant advancement in continuous information theory.
More Related Videos
08:56Automatic Image Processing to Determine the Community Size Structure of Riverine Macroinvertebrates
Published on: January 13, 2023
2.2K
10:27Contrast-Matching Detergent in Small-Angle Neutron Scattering Experiments for Membrane Protein Structural Analysis and Ab Initio Modeling
Published on: October 21, 2018
12.4K
Related Concept Videos
02:39Entropy
30.2K
Salt particles that have dissolved in water never spontaneously come back together in solution to reform solid particles. Moreover, a gas that has expanded in a vacuum remains dispersed and never spontaneously reassembles. The unidirectional nature of these phenomena is the result of a thermodynamic state function called entropy (S). Entropy is the measure of the extent to which the energy is dispersed throughout a system, or in other words, it is proportional to the degree of disorder of a...
30.2K
01:20Entropy and the Second Law of Thermodynamics
2.8K
The second law of thermodynamics can be stated quantitatively using the concept of entropy. Entropy is the measure of disorder of the system.
The relation between entropy and disorder can be illustrated with the example of the phase change of ice to water. In ice, the molecules are located at specific sites giving a solid state, whereas, in a liquid form, these molecules are much freer to move. The molecular arrangement has therefore become more randomized. Although the change in average...
The relation between entropy and disorder can be illustrated with the example of the phase change of ice to water. In ice, the molecules are located at specific sites giving a solid state, whereas, in a liquid form, these molecules are much freer to move. The molecular arrangement has therefore become more randomized. Although the change in average...
2.8K
01:12Sampling Distribution
12.7K
Given simple random samples of size n from a given population with a measured characteristic such as mean, proportion, or standard deviation for each sample, the probability distribution of all the measured characteristics is called a sampling distribution. How much the statistic varies from one sample to another is known as the sampling variability of a statistic. You typically measure the sampling variability of a statistic by its standard error. The standard error of the mean is an example...
12.7K
01:14The Second Law of Thermodynamics
5.3K
In the quest to identify a property that may reliably predict the spontaneity of a process, a promising candidate has been identified: entropy. Scientists refer to the measure of randomness or disorder within a system as entropy. High entropy means high disorder and low energy. To better understand entropy, think of a student’s bedroom. If no energy or work were put into it, the room would quickly become messy. It would exist in a very disordered state, one of high entropy. Energy must be...
5.3K
02:05Entropy and Solvation
7.1K
The process of surrounding a solute with solvent is called solvation. It involves evenly distributing the solute within the solvent. The rule of thumb for determining a solvent for a given compound is that like dissolves like. A good solvent has molecular characteristics similar to those of the compound to be dissolved. For example, polar solutions dissolve polar solutes, and apolar solvents dissolve apolar solutes. A polar solvent is a solvent that has a high dielectric constant (ϵ...
7.1K
02:38Third Law of Thermodynamics
18.9K
A pure, perfectly crystalline solid possessing no kinetic energy (that is, at a temperature of absolute zero, 0 K) may be described by a single microstate, as its purity, perfect crystallinity,and complete lack of motion means there is but one possible location for each identical atom or molecule comprising the crystal (W = 1). According to the Boltzmann equation, the entropy of this system is zero.
18.9K