Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Accuracy and Precision01:52

Accuracy and Precision

12.5K
Scientists typically make repeated measurements of a quantity to ensure the quality of their findings and to evaluate both the precision and the accuracy of their results. Measurements are said to be precise if they yield very similar results when repeated in the same manner. A measurement is considered accurate if it yields a result that is very close to the true or the accepted value. Precise values agree with each other; accurate values agree with a true value.  Highly accurate...
12.5K
Accuracy and Precision01:52

Accuracy and Precision

2.9K
2.9K
Accuracy and Errors in Hypothesis Testing01:13

Accuracy and Errors in Hypothesis Testing

716
Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5%...
716
Receiver Operating Characteristic Plot01:15

Receiver Operating Characteristic Plot

584
A ROC (Receiver Operating Characteristic) plot is a graphical tool used to assess the performance of a binary classification model by illustrating the trade-off between sensitivity (true positive rate) and specificity (false positive rate). By plotting sensitivity against 1 - specificity across various threshold settings, the ROC curve shows how well the model distinguishes between classes, with a curve closer to the top-left corner indicating a more accurate model. The area under the ROC curve...
584
Accuracy, limits, and approximation01:28

Accuracy, limits, and approximation

1.4K
Accuracy, limits, and approximations are common in many fields, especially in engineering calculations. These concepts are imperative for ensuring that a given value is as close as possible to its true value.
Accuracy is defined as the closeness of the measured value to the true or actual value. In engineering mechanics, repeated measurements are taken during theoretical or experimental analyses to ensure that the result is precise and accurate.
The accuracy of any solution is based on the...
1.4K
Improving Translational Accuracy02:07

Improving Translational Accuracy

2.6K
2.6K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

WEMAC: Women and Emotion Multi-modal Affective Computing dataset.

Scientific data·2024
Same author

Assessing Information Transmission in Data Transformations with the Channel Multivariate Entropy Triangle.

Entropy (Basel, Switzerland)·2020
Same author

The Case for Shifting the Rényi Entropy.

Entropy (Basel, Switzerland)·2020
Same author

The Rényi Entropies Operate in Positive Semifields.

Entropy (Basel, Switzerland)·2020
Same author

Interactive knowledge discovery and data mining on genomic expression data with numeric formal concept analysis.

BMC bioinformatics·2016

Related Experiment Videos

100% classification accuracy considered harmful: the normalized information transfer factor explains the accuracy

Francisco J Valverde-Albacete1, Carmen Peláez-Moreno2

  • 1Departamento de Lenguajes y Sistemas Informáticos, Universidad Nacional de Educación a Distancia, Madrid, Spain.

Plos One
|January 16, 2014
PubMed
Summary
This summary is machine-generated.

Accuracy is a flawed measure of predictive model performance. New metrics, entropy-modulated accuracy (EMA) and normalized information transfer (NIT), better assess information transfer and classifier effectiveness.

Related Experiment Videos

Area of Science:

  • Machine Learning
  • Information Theory
  • Computational Neuroscience

Background:

  • Traditional accuracy metrics can be misleading, failing to capture essential information transfer in classification tasks.
  • High accuracy does not always equate to superior predictive power due to a paradox in performance measurement.

Purpose of the Study:

  • To introduce novel metrics for classification performance that account for information learned by classifiers.
  • To address the limitations of accuracy by developing more reliable measures of classifier effectiveness.

Main Methods:

  • Utilized combinatorial analysis and the entropy triangle to visualize classifier performance across different class numbers.
  • Developed entropy-modulated accuracy (EMA) and normalized information transfer (NIT) from first principles.

Main Results:

  • Demonstrated that accuracy can be paradoxical, with higher accuracy not always indicating better predictive power.
  • Introduced EMA as a robust measure of expected accuracy, factoring out input distribution influence.
  • Introduced NIT as a measure of information transmission efficiency and classifier learning effectiveness.

Conclusions:

  • EMA and NIT provide more meaningful and interpretable assessments of classifier performance than traditional accuracy.
  • These new metrics are valuable for evaluating classifiers, especially when information transfer is the primary goal, as shown in a mind-reading task.