Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Sample Size Calculation01:19

Sample Size Calculation

6.2K
Knowledge of the sample size is the first requirement to conduct random sampling or an experiment. The sample size is the total number of units, observations, or groups (in some cases) used to get the data to estimate a population parameter. As the name suggests, the sample size is that of the sample drawn from the population and differs from the population size.
The sample size for the given experiment or sampling effort is fundamental to any study design. Sample size decides the number of...
6.2K
One-Way ANOVA: Unequal Sample Sizes01:15

One-Way ANOVA: Unequal Sample Sizes

6.6K
One-way ANOVA can be performed on three or more samples of unequal sizes. However, calculations get complicated when sample sizes are not always the same. So, while performing ANOVA with unequal samples size, the following equation is used:
6.6K
One-Way ANOVA: Equal Sample Sizes01:15

One-Way ANOVA: Equal Sample Sizes

4.0K
One-Way ANOVA can be performed on three or more samples with equal or unequal sample sizes. When one-way ANOVA is performed on two datasets with samples of equal sizes, it can be easily observed that the computed F statistic is highly sensitive to the sample mean.
Different sample means can result in different values for the variance estimate: variance between samples. This is because the variance between samples is calculated as the product of the sample size and the variance between the...
4.0K
Testing a Claim about Population Proportion01:24

Testing a Claim about Population Proportion

3.9K
A complete procedure for testing a claim about a population proportion is provided here.
There are two methods of testing a claim about a population proportion: (1) Using the sample proportion from the data where a binomial distribution is approximated to the normal distribution and (2) Using the binomial probabilities calculated from the data.
The first method uses normal distribution as an approximation to the binomial distribution. The requirements are as follows: sample size is large...
3.9K
One-Compartment Open Model: Wagner-Nelson and Loo Riegelman Method for ka Estimation01:24

One-Compartment Open Model: Wagner-Nelson and Loo Riegelman Method for ka Estimation

1.1K
This lesson introduces two critical methods in pharmacokinetics, the Wagner-Nelson and Loo-Riegelman methods, used for estimating the absorption rate constant (ka) for drugs administered via non-intravenous routes. The Wagner-Nelson method relates ka to the plasma concentration derived from the slope of a semilog percent unabsorbed time plot. However, it is limited to drugs with one-compartment kinetics and can be impacted by factors like gastrointestinal motility or enzymatic degradation.
On...
1.1K
Prediction Intervals01:03

Prediction Intervals

3.1K
The interval estimate of any variable is known as the prediction interval. It helps decide if a point estimate is dependable.
However, the point estimate is most likely not the exact value of the population parameter, but close to it. After calculating point estimates, we construct interval estimates, called confidence intervals or prediction intervals. This prediction interval comprises a range of values unlike the point estimate and is a better predictor of the observed sample value, y. 
3.1K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Recommendations for Performance Evaluation of Machine Learning in Pathology: A Concept Paper From the College of American Pathologists.

Archives of pathology & laboratory medicine·2023
Same author

Independent assessment of a deep learning system for lymph node metastasis detection on the Augmented Reality Microscope.

Journal of pathology informatics·2023
Same author

Accelerated Progression of Disseminated Coccidioidomycosis Following SARS-CoV-2 Infection: A Case Report.

Military medicine·2021
Same author

Introduction to Artificial Intelligence and Machine Learning for Pathology.

Archives of pathology & laboratory medicine·2021

Related Experiment Video

Updated: Jan 10, 2026

Problem-Solving Before Instruction PS-I: A Protocol for Assessment and Intervention in Students with Different Abilities
10:26

Problem-Solving Before Instruction PS-I: A Protocol for Assessment and Intervention in Students with Different Abilities

Published on: September 11, 2021

4.3K

CONTEST: A generalization of ONEST to estimate sample size for predictive augmented intelligence method validation

Benjamin K Olson1, Joseph H Rosenthal2, Ryan D Kappedal3

  • 1University of California Santa Cruz, Santa Cruz, CA, USA.

Journal of Pathology Informatics
|November 24, 2025
PubMed
Summary

New statistical methods, like CONTEST, can calculate sample sizes for validating subjective clinical tests, especially when machine learning tools lack FDA guidance for multiclass problems. This aids in accurate assay validation for patient care.

Keywords:
Artificial intelligenceComputer visionEffect-sizeImmunohistochemistryMulticlassMultiraterValidation

More Related Videos

Automatic Image Processing to Determine the Community Size Structure of Riverine Macroinvertebrates
08:56

Automatic Image Processing to Determine the Community Size Structure of Riverine Macroinvertebrates

Published on: January 13, 2023

2.8K
Author Spotlight: Addressing Technical and Subjective Challenges in Measuring Classroom Attention
06:37

Author Spotlight: Addressing Technical and Subjective Challenges in Measuring Classroom Attention

Published on: December 15, 2023

5.2K

Related Experiment Videos

Last Updated: Jan 10, 2026

Problem-Solving Before Instruction PS-I: A Protocol for Assessment and Intervention in Students with Different Abilities
10:26

Problem-Solving Before Instruction PS-I: A Protocol for Assessment and Intervention in Students with Different Abilities

Published on: September 11, 2021

4.3K
Automatic Image Processing to Determine the Community Size Structure of Riverine Macroinvertebrates
08:56

Automatic Image Processing to Determine the Community Size Structure of Riverine Macroinvertebrates

Published on: January 13, 2023

2.8K
Author Spotlight: Addressing Technical and Subjective Challenges in Measuring Classroom Attention
06:37

Author Spotlight: Addressing Technical and Subjective Challenges in Measuring Classroom Attention

Published on: December 15, 2023

5.2K

Area of Science:

  • Clinical Diagnostics
  • Biostatistics
  • Medical Informatics

Background:

  • Assay validation is crucial for clinical laboratories before reporting patient results.
  • Current FDA guidance lacks specific frameworks for validating multiclass machine learning decision-support tools.
  • Traditional validation metrics like accuracy, precision, reportable range, and reference intervals are essential.

Purpose of the Study:

  • To introduce Cases and Observers Needed to Evaluate a Subjective Test (CONTEST), an extension of ONEST, for validating subjective tests.
  • To demonstrate a method for calculating the required sample size for test validation using parametric probability distributions.
  • To provide a framework for validating tools that augment subjective tests, particularly in resource-limited settings.

Main Methods:

  • Developed a treatment effect extension of the Observers Needed to Evaluate a Subjective Test (ONEST) framework, named CONTEST.
  • Specified agreement and disagreement distributions using parametric probability distributions.
  • Derived a method for calculating the necessary sample size for test validation based on desired level and power.

Main Results:

  • Demonstrated that sample size calculations for subjective test validation can be performed using CONTEST.
  • Showed that agreement and disagreement distributions can be modeled parametrically.
  • The proposed method is suitable for validating augmented subjective tests using existing datasets.

Conclusions:

  • CONTEST offers a statistically sound approach for calculating sample sizes in subjective test validation.
  • This method addresses the gap in validating multiclass decision-support tools, especially those employing machine learning.
  • CONTEST is particularly valuable for resource-constrained settings needing to validate diagnostic assays.