Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Sample Size Calculation01:19

Sample Size Calculation

3.8K
Knowledge of the sample size is the first requirement to conduct random sampling or an experiment. The sample size is the total number of units, observations, or groups (in some cases) used to get the data to estimate a population parameter. As the name suggests, the sample size is that of the sample drawn from the population and differs from the population size.
The sample size for the given experiment or sampling effort is fundamental to any study design. Sample size decides the number of...
3.8K
Prediction Intervals01:03

Prediction Intervals

2.3K
The interval estimate of any variable is known as the prediction interval. It helps decide if a point estimate is dependable.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y. 
2.3K
Estimating Population Mean with Unknown Standard Deviation01:22

Estimating Population Mean with Unknown Standard Deviation

8.3K
In practice, we rarely know the population standard deviation. In the past, when the sample size was large, this did not present a problem to statisticians. They used the sample standard deviation s as an estimate for σ and proceeded as before to calculate a confidence interval with close enough results. However, statisticians ran into problems when the sample size was small. A small sample size caused inaccuracies in the confidence interval.
William S. Gosset (1876–1937) of the...
8.3K
Margin of Error01:27

Margin of Error

4.5K
The margin of error is also called the maximum error of an estimate. The margin of error is the maximum possible or expected difference between the observed sample parameter value and the actual population parameter value. For proportion, it is the maximum difference between the value of sample proportion obtained from the data and the true value of population proportion. As the true value of the population parameter is not known, the margin of error is calculated using the sample statistic.
4.5K
Receiver Operating Characteristic Plot01:15

Receiver Operating Characteristic Plot

337
A ROC (Receiver Operating Characteristic) plot is a graphical tool used to assess the performance of a binary classification model by illustrating the trade-off between sensitivity (true positive rate) and specificity (false positive rate). By plotting sensitivity against 1 - specificity across various threshold settings, the ROC curve shows how well the model distinguishes between classes, with a curve closer to the top-left corner indicating a more accurate model. The area under the ROC curve...
337
Survival Tree01:19

Survival Tree

166
Survival trees are a non-parametric method used in survival analysis to model the relationship between a set of covariates and the time until an event of interest occurs, often referred to as the "time-to-event" or "survival time." This method is particularly useful when dealing with censored data, where the event has not occurred for some individuals by the end of the study period, or when the exact time of the event is unknown.
 Building a Survival Tree
Constructing a...
166

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Author Correction: The STARD-AI reporting guideline for diagnostic accuracy studies using artificial intelligence.

Nature medicine·2026
Same author

What influences whether researchers adhere to healthcare reporting guidelines successfully? A thematic synthesis.

Research integrity and peer review·2026
Same author

A scoping review of guidance on sharing trial results with patients and the public finds inconsistent recommendations and limited methodological development.

Journal of clinical epidemiology·2026
Same author

Stakeholder perspectives on potential open-label bias in patient-reported outcome results from cancer clinical trials: evidence from the IMPROvE project.

Value in health : the journal of the International Society for Pharmacoeconomics and Outcomes Research·2026
Same author

SPIRIT 2025 statement: updated guideline for protocols of randomised trials.

Lancet (London, England)·2026
Same author

Recommendations for studying the safety, efficacy and durability of intracranial aneurysm devices.

European stroke journal·2026

Related Experiment Video

Updated: Sep 17, 2025

Automatic Image Processing to Determine the Community Size Structure of Riverine Macroinvertebrates
08:56

Automatic Image Processing to Determine the Community Size Structure of Riverine Macroinvertebrates

Published on: January 13, 2023

2.3K

Extended sample size calculations for evaluation of prediction models using a threshold for classification.

Rebecca Whittle1,2, Joie Ensor3,4, Lucinda Archer3,4

  • 1Department of Applied Health Sciences, School of Health Sciences, College of Medicine and Health, University of Birmingham, Birmingham, UK. r.l.whittle@bham.ac.uk.

BMC Medical Research Methodology
|July 2, 2025
PubMed
Summary

Calculating adequate sample size is crucial for accurate individualised risk prediction models. This study provides new formulas and code to determine the minimum sample size for precisely estimating threshold-based performance measures, enhancing model evaluation.

Keywords:
Classification modelsClinical prediction modelsExternal validationModel evaluationPerformance measuresSample sizeThreshold

More Related Videos

An R-Based Landscape Validation of a Competing Risk Model
05:37

An R-Based Landscape Validation of a Competing Risk Model

Published on: September 16, 2022

2.2K
Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances
07:35

Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances

Published on: October 11, 2018

7.6K

Related Experiment Videos

Last Updated: Sep 17, 2025

Automatic Image Processing to Determine the Community Size Structure of Riverine Macroinvertebrates
08:56

Automatic Image Processing to Determine the Community Size Structure of Riverine Macroinvertebrates

Published on: January 13, 2023

2.3K
An R-Based Landscape Validation of a Competing Risk Model
05:37

An R-Based Landscape Validation of a Competing Risk Model

Published on: September 16, 2022

2.2K
Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances
07:35

Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances

Published on: October 11, 2018

7.6K

Area of Science:

  • Biostatistics
  • Medical Informatics
  • Epidemiology

Background:

  • Accurate sample size calculation is essential for reliable individualised risk prediction models.
  • Existing guidance focuses on calibration, discrimination, and net benefit, but often overlooks threshold-based measures.
  • Threshold-based performance metrics are frequently reported in clinical practice.

Purpose of the Study:

  • To extend existing sample size guidance for prediction models.
  • To provide methods for precisely estimating threshold-based performance measures.
  • To offer tools for calculating minimum sample size in external validation studies.

Main Methods:

  • Developed closed-form solutions for sample size estimation of accuracy, specificity, sensitivity, positive predictive value (PPV), and negative predictive value (NPV).
  • Utilized an iterative method for estimating sample size for the F1-score.
  • Incorporated user-defined target standard error, expected performance values, and outcome prevalence.
  • Considered extensions for time-to-event outcomes.

Main Results:

  • New formulae and computational tools (Python, R, Stata) are provided for sample size calculation.
  • In examples, the required sample size for threshold-based measures was often less than for calibration slope estimation.
  • The methods enable precise estimation of various performance metrics in external validation.

Conclusions:

  • The developed methods and tools facilitate accurate sample size determination for threshold-based performance measures.
  • These criteria should complement existing guidance for a comprehensive evaluation of prediction models.
  • Researchers can now more reliably assess the minimum sample size needed for robust model validation.