Search research articles

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

Related Concept Videos

Receiver Operating Characteristic Plot

Receiver Operating Characteristic Plot

A ROC (Receiver Operating Characteristic) plot is a graphical tool used to assess the performance of a binary classification model by illustrating the trade-off between sensitivity (true positive rate) and specificity (false positive rate). By plotting sensitivity against 1 - specificity across various threshold settings, the ROC curve shows how well the model distinguishes between classes, with a curve closer to the top-left corner indicating a more accurate model. The area under the ROC curve...

Sample Size Calculation

Sample Size Calculation

Knowledge of the sample size is the first requirement to conduct random sampling or an experiment. The sample size is the total number of units, observations, or groups (in some cases) used to get the data to estimate a population parameter. As the name suggests, the sample size is that of the sample drawn from the population and differs from the population size.
The sample size for the given experiment or sampling effort is fundamental to any study design. Sample size decides the number of...

Margin of Error

Margin of Error

The margin of error is also called the maximum error of an estimate. The margin of error is the maximum possible or expected difference between the observed sample parameter value and the actual population parameter value. For proportion, it is the maximum difference between the value of sample proportion obtained from the data and the true value of population proportion. As the true value of the population parameter is not known, the margin of error is calculated using the sample statistic.

Accuracy and Errors in Hypothesis Testing

Accuracy and Errors in Hypothesis Testing

Hypothesis testing is a fundamental statistical tool that begins with the assumption that the null hypothesis H0 is true. During this process, two types of errors can occur: Type I and Type II. A Type I error refers to the incorrect rejection of a true null hypothesis, while a Type II error involves the failure to reject a false null hypothesis.
In hypothesis testing, the probability of making a Type I error, denoted as α, is commonly set at 0.05. This significance level indicates a 5% chance...

Sample Proportion and Population Proportion

Sample Proportion and Population Proportion

Collecting samples or responses from an entire population takes significant time and effort, so a researcher collects responses from only a sample of that population. Suppose a study needs to collect information about a specific mobile application. After sample collection, the researcher analyzes the data and discovers that most individuals in the sample use that specific mobile application. The sample proportion measures the number of individuals in a sample who either use or don't use the...

Regression Toward the Mean

Regression Toward the Mean

Regression toward the mean (“RTM”) is a phenomenon in which extremely high or low values—for example, and individual’s blood pressure at a particular moment—appear closer to a group’s average upon remeasuring. Although this statistical peculiarity is the result of random error and chance, it has been problematic across various medical, scientific, financial and psychological applications. In particular, RTM, if not taken into account, can interfere when researchers try to extrapolate results...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same author

Due to potential concerns of bias and conflicts of interest, regulatory bodies should not do evaluation methodology research related to their regulatory missions.

Medical physics·2017

Same author

Erratum: "Evaluating computer-aided detection algorithms".

Medical physics·2017

Same author

A bivariate contaminated binormal model for robust fitting of proper ROC curves to a pair of correlated, possibly degenerate, ROC datasets.

Medical physics·2017

Same author

ROC or FROC? It depends on the research question.

Medical physics·2017

Same author

Multi-Institutional Evaluation of Digital Tomosynthesis, Dual-Energy Radiography, and Conventional Chest Radiography for the Detection and Management of Pulmonary Nodules.

Radiology·2016

Same author

On the meaning of the weighted alternative free-response operating characteristic figure of merit.

Medical physics·2016

Search research articles

Related Experiment Video

Updated: Jun 14, 2026

Advancing Dyslexia Assessment in Children Through Computerized Testing

Advancing Dyslexia Assessment in Children Through Computerized Testing

Published on: August 16, 2024

Prediction accuracy of a sample-size estimation method for ROC studies.

Dev P Chakraborty¹

¹Department of Radiology, University of Pittsburgh, Pittsburgh, PA 15213, USA. dpc10@pitt.edu

Academic Radiology

|April 13, 2010

Summary

This summary is machine-generated.

The Hillis-Berbaum method for sample-size estimation in ROC studies tends to overestimate case numbers. Random-case generalization showed reasonable accuracy, especially with more readers in pilot studies.

More Related Videos

Comparison of Predictive Performance of Three Lymph Node Staging Systems in Colorectal Signet Ring Cell Carcinoma Based on Machine Learning Model

Comparison of Predictive Performance of Three Lymph Node Staging Systems in Colorectal Signet Ring Cell Carcinoma Based on Machine Learning Model

Published on: April 18, 2025

Related Experiment Videos

Last Updated: Jun 14, 2026

Advancing Dyslexia Assessment in Children Through Computerized Testing

Advancing Dyslexia Assessment in Children Through Computerized Testing

Published on: August 16, 2024

Comparison of Predictive Performance of Three Lymph Node Staging Systems in Colorectal Signet Ring Cell Carcinoma Based on Machine Learning Model

Comparison of Predictive Performance of Three Lymph Node Staging Systems in Colorectal Signet Ring Cell Carcinoma Based on Machine Learning Model

Published on: April 18, 2025

Area of Science:

Medical Imaging Analysis
Statistical Methods in Research
Receiver Operating Characteristic (ROC) Studies

Background:

Accurate sample-size estimation is critical for the validity and power of Receiver Operating Characteristic (ROC) studies.
Existing methods for sample-size estimation in ROC studies require careful evaluation of their prediction accuracy.

Purpose of the Study:

To assess the prediction accuracy of the Hillis-Berbaum (HB) sample-size estimation method for ROC studies.
To evaluate the HB method's performance under varying reader and case variability conditions using Monte Carlo simulations.

Main Methods:

Generated pilot datasets using two ROC ratings simulators with distinct reader and case variabilities (Low-High [LH] and High-Low [HL]).
Employed Dorfman-Berbaum-Metz multiple-reader multiple-case (DBM-MRMC) analysis to estimate variances.
Inputted variance estimates into the HB method to predict sample sizes for 80% power, assessing prediction accuracy via a defined index.

Main Results:

The HB method demonstrated reasonable prediction accuracy (approx. 50%) for random-case generalization with 5 readers and 100 cases in pilot studies.
Prediction accuracy was generally higher under LH conditions compared to HL conditions.
Overestimation of cases by the HB method was observed under ideal conditions (many pilot readers), linked to larger modality-reader variance estimates in HL scenarios.

Conclusions:

The HB sample-size estimation method tends to overestimate the required number of cases, particularly when reader variability is high.
Random-case generalization offers reasonable prediction accuracy, with performance improving under LH conditions and with approximately 15 readers in pilot studies.
Study designers should consider comparing HB predictions with other methods and previous study sample sizes, especially when reader variability is substantial.