Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Expected Frequencies in Goodness-of-Fit Tests01:19

Expected Frequencies in Goodness-of-Fit Tests

2.5K
A goodness-of-fit test is conducted to determine whether the observed frequency values are statistically similar to the frequencies expected for the dataset. Suppose the expected frequencies for a dataset are equal such as when predicting the frequency of any number appearing when casting a die. In that case, the expected frequency is the ratio of the total number of observations (n)  to the number of categories (k).
2.5K
Wilcoxon Signed-Ranks Test for Matched Pairs01:09

Wilcoxon Signed-Ranks Test for Matched Pairs

135
The Wilcoxon signed-rank test for matched pairs evaluates the null hypothesis by combining the ranks of differences with their signs. It essentially tests whether the median of the differences in a population of matched pairs is zero. Since the test incorporates more information than the sign test, it generally yields more trustable conclusions. This test also does not require the data to follow a normal distribution, but two conditions must be met for it to be applicable: (1) the data must...
135
Friedman Two-way Analysis of Variance by Ranks01:21

Friedman Two-way Analysis of Variance by Ranks

200
Friedman's Two-Way Analysis of Variance by Ranks is a nonparametric test designed to identify differences across multiple test attempts when traditional assumptions of normality and equal variances do not apply. Unlike conventional ANOVA, which requires normally distributed data with equal variances, Friedman's test is ideal for ordinal or non-normally distributed data, making it particularly useful for analyzing dependent samples, such as matched subjects over time or repeated measures...
200
Goodness-of-Fit Test01:16

Goodness-of-Fit Test

3.4K
The goodness-of-fit test is a type of hypothesis test which determines whether the data "fits" a particular distribution. For example, one may suspect that some anonymous data may fit a binomial distribution. A chi-square test (meaning the distribution for the hypothesis test is chi-square) can be used to determine if there is a fit. The null and alternative hypotheses may be written in sentences or stated as equations or inequalities. The test statistic for a goodness-of-fit test is given as...
3.4K
Sign Test for Matched Pairs01:17

Sign Test for Matched Pairs

135
The sign test for matched pairs offers a robust method for comparing two paired samples, often for the effects of an intervention in one of them. This method is very useful in situations where the underlying distribution of the data is unknown. The test compares two related samples—often pre- and post-treatment measurements on the same subjects—to determine if there are significant differences in their median values.
To conduct the sign test, we first calculate the differences in...
135
One-Way ANOVA: Equal Sample Sizes01:15

One-Way ANOVA: Equal Sample Sizes

3.3K
One-Way ANOVA can be performed on three or more samples with equal or unequal sample sizes. When one-way ANOVA is performed on two datasets with samples of equal sizes, it can be easily observed that the computed F statistic is highly sensitive to the sample mean.
Different sample means can result in different values for the variance estimate: variance between samples. This is because the variance between samples is calculated as the product of the sample size and the variance between the...
3.3K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Deep Learning-Based Prediction of Heart Failure Readmission Using Longitudinal Cardiac Device Data.

Journal of cardiovascular electrophysiology·2026
Same author

Impact of COVID-19 pandemic on the size of US transplant waiting lists.

Clinical transplantation·2022
Same author

Outcomes of living liver donor candidate evaluations in the Living Donor Collective pilot registry.

Clinical transplantation·2021
Same author

In vitro modeling accurately predicts cardiac lead fracture at 10 years.

Heart rhythm·2021
Same author

Outcomes of Living Kidney Donor Candidate Evaluations in the Living Donor Collective Pilot Registry.

Transplantation direct·2021
Same author

Mortality among solid organ waitlist candidates during COVID-19 in the United States.

American journal of transplantation : official journal of the American Society of Transplantation and the American Society of Transplant Surgeons·2021

Related Experiment Video

Updated: Jul 9, 2025

A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments
08:12

A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments

Published on: March 1, 2022

2.5K

Discordant pair analysis for sample efficient model evaluation.

Donald Musgrove1, Andrew Radtke2, Tarek Haddad2

  • 1Medtronic Inc., 8200 Coral Sea St NE, Mounds View, MN, 55112, USA. donald.r.musgrove@medtronic.com.

Scientific Reports
|November 28, 2023
PubMed
Summary

We developed a new discordant pair analysis technique to efficiently assess classification algorithm performance. This method significantly reduces human review by over 90% while maintaining high accuracy and precision.

More Related Videos

Generating Strictly Controlled Stimuli for Figure Recognition Experiments
05:39

Generating Strictly Controlled Stimuli for Figure Recognition Experiments

Published on: March 18, 2019

5.3K
Development of an Individual-Tree Basal Area Increment Model using a Linear Mixed-Effects Approach
04:35

Development of an Individual-Tree Basal Area Increment Model using a Linear Mixed-Effects Approach

Published on: July 3, 2020

3.4K

Related Experiment Videos

Last Updated: Jul 9, 2025

A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments
08:12

A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments

Published on: March 1, 2022

2.5K
Generating Strictly Controlled Stimuli for Figure Recognition Experiments
05:39

Generating Strictly Controlled Stimuli for Figure Recognition Experiments

Published on: March 18, 2019

5.3K
Development of an Individual-Tree Basal Area Increment Model using a Linear Mixed-Effects Approach
04:35

Development of an Individual-Tree Basal Area Increment Model using a Linear Mixed-Effects Approach

Published on: July 3, 2020

3.4K

Area of Science:

  • Computer Science
  • Machine Learning
  • Data Science

Background:

  • Evaluating classification algorithm performance traditionally requires extensive human review of large datasets.
  • Existing methods can be time-consuming, costly, and prone to human error, impacting evaluation precision.

Purpose of the Study:

  • To introduce a novel, computationally efficient technique for assessing classification algorithm effectiveness.
  • To reduce the need for human adjudication in performance evaluation while maintaining or improving accuracy.

Main Methods:

  • The discordant pair analysis technique compares a target algorithm against a baseline algorithm on an unlabeled dataset.
  • Performance estimates are derived solely from the subset of examples where the two algorithms produce discordant classifications.
  • Assumes a known performance baseline and an estimated class distribution for the dataset.

Main Results:

  • The discordant pair method drastically reduces the number of required human adjudications by over 90%.
  • Maintains equivalent levels of sensitivity and specificity compared to traditional exhaustive evaluation methods.
  • Demonstrates potential for improved evaluation quality by minimizing human error.

Conclusions:

  • Discordant pair analysis offers a computationally efficient and precise alternative for evaluating classification algorithms.
  • This technique significantly streamlines the performance assessment process, making it more scalable and cost-effective.
  • The method holds promise for enhancing the reliability of algorithm performance estimates in machine learning workflows.