Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Receiver Operating Characteristic Plot01:15

Receiver Operating Characteristic Plot

150
A ROC (Receiver Operating Characteristic) plot is a graphical tool used to assess the performance of a binary classification model by illustrating the trade-off between sensitivity (true positive rate) and specificity (false positive rate). By plotting sensitivity against 1 - specificity across various threshold settings, the ROC curve shows how well the model distinguishes between classes, with a curve closer to the top-left corner indicating a more accurate model. The area under the ROC curve...
150
Comparing the Survival Analysis of Two or More Groups01:20

Comparing the Survival Analysis of Two or More Groups

179
Survival analysis is a cornerstone of medical research, used to evaluate the time until an event of interest occurs, such as death, disease recurrence, or recovery. Unlike standard statistical methods, survival analysis is particularly adept at handling censored data—instances where the event has not occurred for some participants by the end of the study or remains unobserved. To address these unique challenges, specialized techniques like the Kaplan-Meier estimator, log-rank test, and...
179
Introduction to Nonparametric Statistics01:28

Introduction to Nonparametric Statistics

710
Nonparametric statistics offer a powerful alternative to traditional parametric methods, useful when assumptions about the population distribution cannot be made. Unlike parametric tests, which require data to follow a specific distribution with well-defined parameters (such as the mean and standard deviation), nonparametric tests do not require such constraints. This makes them particularly valuable when dealing with small sample sizes, skewed data, or ordinal and categorical variables.
One of...
710
Statistical Inference Techniques in Hypothesis Testing: Parametric Versus Nonparametric Data01:16

Statistical Inference Techniques in Hypothesis Testing: Parametric Versus Nonparametric Data

125
Statistical inference techniques, paramount in hypothesis testing, differentiate into two broad categories: parametric and nonparametric statistics.
Parametric statistics, as the name suggests, assumes that data follow a specific distribution, often a normal distribution. This assumption enables robust hypothesis testing and estimation. Parametric methods, like the Student's t-test or Goodness-of-fit test, are frequently employed in biostatistics due to their robustness. For instance,...
125
Statistical Methods to Analyze Parametric Data: Student t-Test and Goodness-of-Fit Test01:09

Statistical Methods to Analyze Parametric Data: Student t-Test and Goodness-of-Fit Test

1.6K
In parametric statistics, two fundamental tests stand out for their utility and wide application: the Student's t-test and goodness-of-fit tests. These tests provide researchers with a robust method for drawing insights from data, testing hypotheses, and making informed decisions based on their findings.
The Student's t-test is a statistical test that examines if there is a statistically significant difference between the means of two groups. This test is instrumental when dealing with...
1.6K
Multiple Comparison Tests01:13

Multiple Comparison Tests

3.9K
Multiple comparison test, abbreviated as MCT, is a post hoc analysis generally performed after comparing multiple samples with one or more tests. An MCT will help identify a significantly different sample among multiple samples or a factor among multiple factors.
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
3.9K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Tension pneumoperitoneum combined with CO<sub>2</sub> gas embolism during peroral endoscopic myotomy: a case report and review of literature.

Frontiers in medicine·2026
Same author

Maintenance treatment and survival in patients with newly diagnosed diffuse large B cell lymphoma in the immunochemotherapy era: a systematic review and network meta-analysis.

Clinical & translational oncology : official publication of the Federation of Spanish Oncology Societies and of the National Cancer Institute of Mexico·2026
Same author

Hyperpolarized Molecular Nuclear Spins Achieve Magnetic Amplification.

Physical review letters·2026
Same author

A deep learning PET/CT biomarker for early progression (POD24) and survival stratification in follicular lymphoma: a multicenter study.

European journal of nuclear medicine and molecular imaging·2026
Same author

Zero- to ultralow-field J-spectroscopy with a diamond magnetometer.

Communications chemistry·2026
Same author

Association between body mass index and outcomes in lymphoma-associated haemophagocytic lymphohistiocytosis: A retrospective multicentre cohort study of Jiangsu Cooperative Lymphoma Group (JCLG).

British journal of haematology·2026
Same journal

CrunchLLM: Multitask LLMs for Structured Business Reasoning and Outcome Prediction.

Neurocomputing·2026
Same journal

Deep Learning for analyzing chaotic dynamics in biological time series: Insights from frog heart signals.

Neurocomputing·2026
Same journal

SymRefine: A symbolic regression approach for refining and compressing neural networks.

Neurocomputing·2026
Same journal

Artificial intelligence without restriction surpassing human intelligence with probability one: Theoretical insight into secrets of the brain with AI twins of the brain.

Neurocomputing·2025
Same journal

ShaderNN: A Lightweight and Efficient Inference Engine for Real-time Applications on Mobile GPUs.

Neurocomputing·2025
Same journal

Improving Adversarial Robustness of Deep Neural Networks via Adaptive Margin Evolution.

Neurocomputing·2023
See all related articles
  1. Home
  3. Comparing Multi-class Classifier Performance By Multi-class Roc Analysis: A Nonparametric Approach.
  1. Home
  3. Comparing Multi-class Classifier Performance By Multi-class Roc Analysis: A Nonparametric Approach.

Related Experiment Video

Detection of Architectural Distortion in Prior Mammograms via Analysis of Oriented Patterns
13:44

Detection of Architectural Distortion in Prior Mammograms via Analysis of Oriented Patterns

Published on: August 30, 2013

42.9K

Comparing multi-class classifier performance by multi-class ROC analysis: A nonparametric approach.

Jingyan Xu1

  • 1Department of Radiology, Johns Hopkins University, MD, USA.

Neurocomputing
|April 22, 2024

View abstract on PubMed

Summary
This summary is machine-generated.

This study introduces a new method to estimate the variance of multi-class area under the ROC curve (MAUC) for machine learning classifiers. The approach accurately quantifies classifier performance and aids in comparing multiple models.

Keywords:
Ustatisticsarea under the ROC curve (AUC)jackknifemulti-class AUCmulti-class classificationreceiver operating characteristic (ROC)

More Related Videos

Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances
07:35

Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances

Published on: October 11, 2018

7.5K
Author Spotlight: Validation of SICOLE-R for Assessing Cognitive and Reading Skills in Spanish-Speaking Children and Its Role in Personalized Education
09:00

Author Spotlight: Validation of SICOLE-R for Assessing Cognitive and Reading Skills in Spanish-Speaking Children and Its Role in Personalized Education

Published on: August 16, 2024

758

Related Experiment Videos

Detection of Architectural Distortion in Prior Mammograms via Analysis of Oriented Patterns
13:44

Detection of Architectural Distortion in Prior Mammograms via Analysis of Oriented Patterns

Published on: August 30, 2013

42.9K
Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances
07:35

Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances

Published on: October 11, 2018

7.5K
Author Spotlight: Validation of SICOLE-R for Assessing Cognitive and Reading Skills in Spanish-Speaking Children and Its Role in Personalized Education
09:00

Author Spotlight: Validation of SICOLE-R for Assessing Cognitive and Reading Skills in Spanish-Speaking Children and Its Role in Personalized Education

Published on: August 16, 2024

758

Area of Science:

  • Machine Learning and Statistical Analysis
  • Computational Statistics
  • Pattern Recognition

Background:

  • The area under the ROC curve (AUC) is a standard metric for binary classification performance.
  • Real-world applications frequently involve multi-class classification, necessitating metrics beyond binary AUC.
  • Existing multi-class AUC (MAUC) variance estimation often relies on computationally intensive resampling techniques due to complex correlation patterns.

Purpose of the Study:

  • To generalize DeLong's non-parametric approach for binary AUC variance estimation to multi-class AUC (MAUC).
  • To develop an accurate and efficient method for estimating the variance of MAUC and the covariance of correlated MAUCs.
  • To provide a computationally tractable solution for comparing multi-class classifiers.

Main Methods:

  • Derived a closed-form expression for the covariance matrix of pairwise AUCs within a single MAUC.
  • Obtained an approximate covariance matrix with a compact, matrix factorization form by dropping higher-order terms.
  • Extended the approach to estimate the covariance of correlated MAUCs from competing multi-class classifiers.

Main Results:

  • The proposed method provides accurate variance and covariance estimates for MAUC, confirmed by numerical studies.
  • The derived covariance matrix offers a computationally efficient basis for MAUC variance estimation.
  • For binary correlated AUCs, the results align with DeLong's established method, validating the generalization.

Conclusions:

  • The developed method offers a statistically sound and computationally efficient alternative to resampling for MAUC variance estimation.
  • This work facilitates more reliable quantification and comparison of multi-class classifiers in machine learning and statistical analysis.
  • Source code is available on GitHub for broad adoption and implementation.