Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Entropy and the Second Law of Thermodynamics01:20

Entropy and the Second Law of Thermodynamics

The second law of thermodynamics can be stated quantitatively using the concept of entropy. Entropy is the measure of disorder of the system.
The relation  between entropy and disorder can be illustrated with the example of the phase change of ice to water. In ice, the molecules are located at specific sites giving a solid state, whereas, in a liquid form, these molecules are much freer to move. The molecular arrangement has therefore become more randomized. Although the change in average...
Entropy and the Second Law of Thermodynamics01:26

Entropy and the Second Law of Thermodynamics

Consider an isolated system in which a hot object is placed in contact with a cold one. This is an irreversible process that eventually leads both objects to reach the same equilibrium temperature. It is crucial to note that the constituents of any substance exhibit increased disorder at higher temperatures. As a cold substance absorbs heat, its constituents become more disordered. The energy transfer from a hotter object to a cooler one increases the system's disorder or randomness. This...
Entropy02:39

Entropy

Salt particles that have dissolved in water never spontaneously come back together in solution to reform solid particles. Moreover, a gas that has expanded in a vacuum remains dispersed and never spontaneously reassembles. The unidirectional nature of these phenomena is the result of a thermodynamic state function called entropy (S). Entropy is the measure of the extent to which the energy is dispersed throughout a system, or in other words, it is proportional to the degree of disorder of a...
Entropy01:18

Entropy

The first law of thermodynamics is quantitatively formulated via an equation relating the internal energy of a system, the heat exchanged by it, and the work done on it. A quantitative formulation of the second law of thermodynamics leads to defining a state function, the entropy.
When an ideal gas expands isothermally, the disorder in the gas increases. From the molecular perspective, the gas molecules have more volume to move around in.
Consider an infinitesimal step in the expansion, which...
Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving01:29

Mechanistic Models: Compartment Models in Algorithms for Numerical Problem Solving

Mechanistic models play a crucial role in algorithms for numerical problem-solving, particularly in nonlinear mixed effects modeling (NMEM). These models aim to minimize specific objective functions by evaluating various parameter estimates, leading to the development of systematic algorithms. In some cases, linearization techniques approximate the model using linear equations.
In individual population analyses, different algorithms are employed, such as Cauchy's method, which uses a...
Expected Frequencies in Goodness-of-Fit Tests01:19

Expected Frequencies in Goodness-of-Fit Tests

A goodness-of-fit test is conducted to determine whether the observed frequency values are statistically similar to the frequencies expected for the dataset. Suppose the expected frequencies for a dataset are equal such as when predicting the frequency of any number appearing when casting a die. In that case, the expected frequency is the ratio of the total number of observations (n) to the number of categories (k).

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Edge-Centric Embeddings of Digraphs: Properties and Stability Under Sparsification.

Entropy (Basel, Switzerland)·2025
Same author

Feeding Ecology of the Cuvier's Gazelle (<i>Gazella cuvieri</i>, Ogilby, 1841) in the Sahara Desert.

Animals : an open access journal from MDPI·2023
Same author

Publisher Correction: Identifying priority conservation areas in a Saharan environment by highlighting the endangered Cuvier's Gazelle as a flagship species.

Scientific reports·2020
Same author

Identifying priority conservation areas in a Saharan environment by highlighting the endangered Cuvier's Gazelle as a flagship species.

Scientific reports·2020
Same author

Aerial obstacle detection with 3-D mobile devices.

IEEE journal of biomedical and health informatics·2014
Same author

Increasing the contrast of the brain MR FLAIR images using fuzzy membership functions and structural similarity indices in order to segment MS lesions.

PloS one·2013

Related Experiment Video

Updated: Jun 20, 2026

A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments
08:12

A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments

Published on: March 1, 2022

Learning Gaussian mixture models with entropy-based criteria.

Antonio Penalver Benavent1, Francisco Escolano Ruiz, Juan Manuel Saez

  • 1Departamento de Estadística, Matemáticas e Informática, Universidad Miguel Hernández, Elche 03202, Spain. a.penalver@umh.es

IEEE Transactions on Neural Networks
|September 23, 2009
PubMed
Summary
This summary is machine-generated.

This study introduces an entropy-based Expectation-Maximization (EM) algorithm to improve Gaussian mixture model parameter estimation. The novel approach optimizes component selection and offers robust solutions for density estimation and classification tasks.

More Related Videos

Applications of EEG Neuroimaging Data: Event-related Potentials, Spectral Power, and Multiscale Entropy
11:15

Applications of EEG Neuroimaging Data: Event-related Potentials, Spectral Power, and Multiscale Entropy

Published on: June 27, 2013

Large-scale Reconstructions and Independent, Unbiased Clustering Based on Morphological Metrics to Classify Neurons in Selective Populations
12:27

Large-scale Reconstructions and Independent, Unbiased Clustering Based on Morphological Metrics to Classify Neurons in Selective Populations

Published on: February 15, 2017

Related Experiment Videos

Last Updated: Jun 20, 2026

A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments
08:12

A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments

Published on: March 1, 2022

Applications of EEG Neuroimaging Data: Event-related Potentials, Spectral Power, and Multiscale Entropy
11:15

Applications of EEG Neuroimaging Data: Event-related Potentials, Spectral Power, and Multiscale Entropy

Published on: June 27, 2013

Large-scale Reconstructions and Independent, Unbiased Clustering Based on Morphological Metrics to Classify Neurons in Selective Populations
12:27

Large-scale Reconstructions and Independent, Unbiased Clustering Based on Morphological Metrics to Classify Neurons in Selective Populations

Published on: February 15, 2017

Area of Science:

  • Machine Learning
  • Statistical Modeling
  • Computational Statistics

Background:

  • Gaussian mixture models (GMMs) are widely used for density estimation and clustering.
  • The standard Expectation-Maximization (EM) algorithm for GMMs can be sensitive to initial parameters and may converge to suboptimal solutions.
  • Determining the optimal number of components (kernels) in a GMM is a significant challenge.

Purpose of the Study:

  • To develop a robust method for estimating Gaussian mixture model parameters, addressing the limitations of the standard EM algorithm.
  • To introduce a novel approach for determining the optimal number of components in a GMM.
  • To propose entropy-based criteria for evaluating GMM quality and guiding model selection.

Main Methods:

  • Introduced the use of probability density function (pdf) entropy for each kernel to assess GMM quality.
  • Developed two methods for approximating kernel entropy.
  • Modified the EM algorithm to find the optimal number of mixture components.
  • Implemented two stopping criteria: Gaussianity Deficiency (GD) and Minimum Description Length (MDL).
  • Proposed the entropy-based EM (EBEM) algorithm, which starts with one kernel and uses splitting based on GD.

Main Results:

  • The EBEM algorithm effectively estimates GMM parameters and determines the optimal number of components.
  • Experimental results demonstrated superior performance in probability density estimation, pattern classification, and color image segmentation compared to existing methods.
  • The proposed Gaussianity Deficiency (GD) criterion proved effective for model order selection.

Conclusions:

  • The entropy-based EM (EBEM) algorithm offers a significant improvement over traditional methods for GMM parameter estimation and model selection.
  • EBEM provides a robust and efficient solution for tasks requiring accurate density estimation and classification.
  • The developed entropy-based criteria, particularly GD, enhance the reliability of GMMs in complex applications.