Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Receiver Operating Characteristic Plot01:15

Receiver Operating Characteristic Plot

A ROC (Receiver Operating Characteristic) plot is a graphical tool used to assess the performance of a binary classification model by illustrating the trade-off between sensitivity (true positive rate) and specificity (false positive rate). By plotting sensitivity against 1 - specificity across various threshold settings, the ROC curve shows how well the model distinguishes between classes, with a curve closer to the top-left corner indicating a more accurate model. The area under the ROC curve...
Margin of Error01:27

Margin of Error

The margin of error is also called the maximum error of an estimate. The margin of error is the maximum possible or expected difference between the observed sample parameter value and the actual population parameter value. For proportion, it is the maximum difference between the value of sample proportion obtained from the data and the true value of population proportion. As the true value of the population parameter is not known, the margin of error is calculated using the sample statistic.
Statistical Analysis: Overview01:11

Statistical Analysis: Overview

When we take repeated measurements on the same or replicated samples, we will observe inconsistencies in the magnitude. These inconsistencies are called errors. To categorize and characterize these results and their errors, the researcher can use statistical analysis to determine the quality of the measurements and/or suitability of the methods.
One of the most commonly used statistical quantifiers is the mean, which is the ratio between the sum of the numerical values of all results and the...
Region of Convergence of Laplace Tarnsform01:20

Region of Convergence of Laplace Tarnsform

The Region of Convergence (ROC) is a fundamental concept in signal processing and system analysis, particularly associated with the Laplace transform. The ROC represents an area in the complex plane where the Laplace transform of a given signal converges, determining the transform's applicability and utility.
Consider a decaying exponential signal that begins at a specific time. When deriving its Laplace transform, the time-domain variable is replaced with a complex variable. This substitution...
Contaminants and Errors01:16

Contaminants and Errors

Effective sample preparation is crucial for accurate and reliable laboratory analysis. During this process, two significant sources of error can arise: concentration bias from improper sample splitting and contamination caused by methods used to reduce particle size, such as grinding or homogenization. Identifying and minimizing these potential errors is crucial to ensuring the validity of the analysis.
Another key consideration is determining the appropriate number of samples required to...
Confidence Intervals01:21

Confidence Intervals

An unbiased point estimate is often insufficient to predict a population estimate, such as population mean or population proportion. In this scenario, a confidence interval is used. A confidence interval is an estimate similar to a sample proportion. However, unlike the point estimate which is a single value, the confidence interval contains a range of values. These values have lower and upper limits, known as confidence limits, and can be designated as L1 and L2, respectively.
A confidence...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Learning inherent genetic patterns and trait associations with deep generative models for discrete genotype simulation.

GigaScienceĀ·2026
Same author

Investigating the molecular mechanisms of resveratrol in treating diabetic foot ulcers: a comprehensive analysis of network pharmacology and experiment validation.

Frontiers in molecular biosciencesĀ·2025
Same author

Self-supervised representation learning on gene expression data.

Bioinformatics (Oxford, England)Ā·2025
Same author

Performance of a first-trimester combined screening for preterm preeclampsia in the United States population using the fetal medicine foundation competing risks model.

American journal of obstetrics & gynecology MFMĀ·2025
Same author

AuNRs-PPARĪ³mAb Induce Targeted Adipocyte Apoptosis Through Photothermal Effects for Effective Localized Fat Reduction.

International journal of nanomedicineĀ·2025
Same author

Modulation of COVID-19 incidence by environmental stressors is variant between pre-Omicron and Omicron periods.

Scientific reportsĀ·2025

Related Experiment Video

Updated: Jun 16, 2026

Advancing Dyslexia Assessment in Children Through Computerized Testing
09:00

Advancing Dyslexia Assessment in Children Through Computerized Testing

Published on: August 16, 2024

Small-sample precision of ROC-related estimates.

Blaise Hanczar1, Jianping Hua, Chao Sima

  • 1LIPADE, University Paris Descartes, Paris, France.

Bioinformatics (Oxford, England)
|February 5, 2010
PubMed
Summary

Estimates of receiver operator characteristic (ROC) curve metrics like area under the curve (AUC), true positive rate (TPR), and false positive rate (FPR) are unreliable, especially with small samples. Even large samples show weak correlation between estimated and true ROC metrics.

More Related Videos

Precision of In Vivo Quantitative Tooth Wear Measurement Using Intra-Oral Scans
09:10

Precision of In Vivo Quantitative Tooth Wear Measurement Using Intra-Oral Scans

Published on: July 12, 2022

Comparison of Predictive Performance of Three Lymph Node Staging Systems in Colorectal Signet Ring Cell Carcinoma Based on Machine Learning Model
07:13

Comparison of Predictive Performance of Three Lymph Node Staging Systems in Colorectal Signet Ring Cell Carcinoma Based on Machine Learning Model

Published on: April 18, 2025

Related Experiment Videos

Last Updated: Jun 16, 2026

Advancing Dyslexia Assessment in Children Through Computerized Testing
09:00

Advancing Dyslexia Assessment in Children Through Computerized Testing

Published on: August 16, 2024

Precision of In Vivo Quantitative Tooth Wear Measurement Using Intra-Oral Scans
09:10

Precision of In Vivo Quantitative Tooth Wear Measurement Using Intra-Oral Scans

Published on: July 12, 2022

Comparison of Predictive Performance of Three Lymph Node Staging Systems in Colorectal Signet Ring Cell Carcinoma Based on Machine Learning Model
07:13

Comparison of Predictive Performance of Three Lymph Node Staging Systems in Colorectal Signet Ring Cell Carcinoma Based on Machine Learning Model

Published on: April 18, 2025

Area of Science:

  • Biostatistics
  • Machine Learning
  • Bioinformatics

Background:

  • Receiver Operator Characteristic (ROC) curves are vital for evaluating discriminant performance in biomedical research.
  • Key metrics include Area Under the Curve (AUC), True Positive Rate (TPR), and False Positive Rate (FPR).
  • Estimating these metrics from small sample sizes presents a significant challenge.

Purpose of the Study:

  • To assess the accuracy of estimated ROC metrics (AUC, TPR, FPR) compared to true values.
  • To investigate the reliability of ROC analysis in small sample scenarios.
  • To evaluate the impact of different classification and error estimation methods on ROC metric reliability.

Main Methods:

  • Simulation studies using data models and real microarray data.
  • Analysis of classification rules including linear discriminant analysis and Support Vector Machines (SVM).
  • Application of various error estimation techniques such as resubstitution, cross-validation, and bootstrap resampling.

Main Results:

  • Considerable root mean square differences between estimated and true ROC metrics were observed, particularly with small samples.
  • Weak correlations were found between true and estimated ROC metrics, even with large sample sizes.
  • Weak regression of true metrics on estimated metrics indicates poor predictive power of estimates.

Conclusions:

  • The study highlights the unreliability of estimated ROC metrics, especially AUC, TPR, and FPR, in small sample settings.
  • Resampling methods, while common, can lead to unreliable published ROC results.
  • Caution is advised when interpreting ROC analysis results derived from limited data or specific estimation techniques.