Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Video

Updated: Nov 27, 2025

Cross-Modal Multivariate Pattern Analysis
13:51

Cross-Modal Multivariate Pattern Analysis

Published on: November 9, 2011

20.3K

Deconstructing Cross-Entropy for Probabilistic Binary Classifiers.

Daniel Ramos1, Javier Franco-Pedroso1, Alicia Lozano-Diez1

  • 1AuDIaS-Audio, Data Intelligence and Speech, Escuela Politecnica Superior, Universidad Autonoma de Madrid, Calle Francisco Tomas y Valiente 11, 28049 Madrid, Spain.

Entropy (Basel, Switzerland)
|December 3, 2020
PubMed
Summary
This summary is machine-generated.

Related Concept Videos

Binomial Probability Distribution01:15

Binomial Probability Distribution

14.7K
A binomial distribution is a probability distribution for a procedure with a fixed number of trials, where each trial can have only two outcomes.
The outcomes of a binomial experiment fit a binomial probability distribution. A statistical experiment can be classified as a binomial experiment if the following conditions are met:
There are a fixed number of trials. Think of trials as repetitions of an experiment. The letter n denotes the number of trials.
There are only two possible outcomes,...
14.7K
Probability in Statistics01:14

Probability in Statistics

20.6K
Probability is the likelihood of an event occurring. The term event is defined as a collection of results of a procedure. An event is a simple event when an outcome cannot be divided into simpler parts.
An example of a simple event is a coin toss. The result of a coin toss is either a head or a tail. Here, head and tail are two simple events. These two simple events make up the sample space. Further, the probability of an event occurring falls within the range of 0 to 1. The probability of an...
20.6K
Propagation of Uncertainty from Random Error00:59

Propagation of Uncertainty from Random Error

1.5K
An experiment often consists of more than a single step. In this case, measurements at each step give rise to uncertainty. Because the measurements occur in successive steps, the uncertainty in one step necessarily contributes to that in the subsequent step. As we perform statistical analysis on these types of experiments, we must learn to account for the propagation of uncertainty from one step to the next. The propagation of uncertainty depends on the type of arithmetic operation performed on...
1.5K
Probability Distributions01:32

Probability Distributions

11.2K
 The probability of a random variable x  is the likelihood of its occurrence. A probability distribution represents the probabilities of a random variable using a formula, graph, or table. There are two types of probability distribution– discrete probability distribution and continuous probability distribution.
A discrete probability distribution is a probability distribution of discrete random variables. It can be categorized into binomial probability distribution and Poisson...
11.2K
Receiver Operating Characteristic Plot01:15

Receiver Operating Characteristic Plot

400
A ROC (Receiver Operating Characteristic) plot is a graphical tool used to assess the performance of a binary classification model by illustrating the trade-off between sensitivity (true positive rate) and specificity (false positive rate). By plotting sensitivity against 1 - specificity across various threshold settings, the ROC curve shows how well the model distinguishes between classes, with a curve closer to the top-left corner indicating a more accurate model. The area under the ROC curve...
400
Prediction Intervals01:03

Prediction Intervals

2.8K
The interval estimate of any variable is known as the prediction interval. It helps decide if a point estimate is dependable.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y. 
2.8K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Oxidative Stress and Inflammation in Hypoxemic Respiratory Diseases and Their Comorbidities: Molecular Insights and Diagnostic Advances in Chronic Obstructive Pulmonary Disease and Sleep Apnea.

Antioxidants (Basel, Switzerland)·2025
Same author

The use of LA-ICP-MS databases to calculate likelihood ratios for the forensic analysis of glass evidence.

Talanta·2018
Same author

An analysis of the influence of deep neural network (DNN) topology in bottleneck feature based language recognition.

PloS one·2017
Same author

Gaussian Mixture Models of Between-Source Variation for Likelihood Ratio Computation from Multivariate Data.

PloS one·2016
Same journal

Research on a Regional Availability Evaluation Model for Road-Area High-Entropy Energy Based on Synergy Factors.

Entropy (Basel, Switzerland)·2026
Same journal

Atmospheric Turbulence Channel Modeling and Performance Analysis of a CO-ZP-OFDM Coherent Optical Communication System for UAV Air-to-Ground Scenarios.

Entropy (Basel, Switzerland)·2026
Same journal

Information Geometry and Asymptotic Theory for SMML Estimators.

Entropy (Basel, Switzerland)·2026
Same journal

Correlation Entropy and Power-Law Kinetics.

Entropy (Basel, Switzerland)·2026
Same journal

Research on the Contagion of Systemic Financial Risk Under the Impact of Climate Risks-From the Perspective of Complex Networks and Machine Learning.

Entropy (Basel, Switzerland)·2026
Same journal

The Statistical-Mechanical Meaning of the Wave Function of Quantum Mechanics.

Entropy (Basel, Switzerland)·2026
See all related articles

This study analyzes cross-entropy, a key metric for classifiers, by linking it to Bayesian decision theory and information theory. It introduces a novel decomposition and an Empirical Cross-Entropy (ECE) plot for better classifier performance analysis.

Area of Science:

  • Machine Learning
  • Information Theory
  • Decision Theory

Background:

  • Cross-entropy is a fundamental metric in machine learning for evaluating classifiers.
  • Its theoretical underpinnings, particularly within Bayesian decision theory and information theory, warrant deeper investigation.

Purpose of the Study:

  • To provide a comprehensive analysis of the cross-entropy function from information-theoretical and Bayesian decision theory perspectives.
  • To introduce a novel decomposition of cross-entropy and a visualization tool for classifier evaluation.

Main Methods:

  • Contextualization of cross-entropy within Bayesian decision theory.
  • Information-theoretical analysis of cross-entropy components, including prior knowledge and feature value (likelihood ratio).
  • Development and application of the Empirical Cross-Entropy (ECE) plot for performance analysis.
Keywords:
BayesianECE plotcalibrationclassifiercross-entropydiscriminationprobabilistic

More Related Videos

Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances
07:35

Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances

Published on: October 11, 2018

7.8K
A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments
08:12

A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments

Published on: March 1, 2022

2.8K

Related Experiment Videos

Last Updated: Nov 27, 2025

Cross-Modal Multivariate Pattern Analysis
13:51

Cross-Modal Multivariate Pattern Analysis

Published on: November 9, 2011

20.3K
Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances
07:35

Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances

Published on: October 11, 2018

7.8K
A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments
08:12

A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments

Published on: March 1, 2022

2.8K

Main Results:

  • Explicit analysis of prior knowledge and feature value contributions to cross-entropy.
  • Introduction of a discrimination-calibration decomposition for precise classifier performance measurement.
  • Demonstration of ECE plots' efficacy in speaker verification and forensic glass analysis.

Conclusions:

  • The study offers new insights into cross-entropy's meaning and interpretation.
  • The discrimination-calibration decomposition enhances classifier evaluation and probability calibration strategies.
  • ECE plots provide a powerful tool for visualizing and analyzing classifier performance.