Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Sample Size Calculation01:19

Sample Size Calculation

3.2K
Knowledge of the sample size is the first requirement to conduct random sampling or an experiment. The sample size is the total number of units, observations, or groups (in some cases) used to get the data to estimate a population parameter. As the name suggests, the sample size is that of the sample drawn from the population and differs from the population size.
The sample size for the given experiment or sampling effort is fundamental to any study design. Sample size decides the number of...
3.2K
Statistical Significance01:50

Statistical Significance

20.1K
Once data is collected from both the experimental and the control groups, a statistical analysis is conducted to find out if there are meaningful differences between the two groups. A statistical analysis determines how likely any difference found is due to chance (and thus not meaningful). In psychology, group differences are considered meaningful, or significant, if the odds that these differences occurred by chance alone are 5 percent or less. Stated another way, if we repeated this...
20.1K
One-Way ANOVA: Equal Sample Sizes01:15

One-Way ANOVA: Equal Sample Sizes

3.1K
One-Way ANOVA can be performed on three or more samples with equal or unequal sample sizes. When one-way ANOVA is performed on two datasets with samples of equal sizes, it can be easily observed that the computed F statistic is highly sensitive to the sample mean.
Different sample means can result in different values for the variance estimate: variance between samples. This is because the variance between samples is calculated as the product of the sample size and the variance between the...
3.1K
Contaminants and Errors01:16

Contaminants and Errors

81
Effective sample preparation is crucial for accurate and reliable laboratory analysis. During this process, two significant sources of error can arise: concentration bias from improper sample splitting and contamination caused by methods used to reduce particle size, such as grinding or homogenization. Identifying and minimizing these potential errors is crucial to ensuring the validity of the analysis.
Another key consideration is determining the appropriate number of samples required to...
81
Significance Testing: Overview01:04

Significance Testing: Overview

3.3K
Significance testing is a set of statistical methods used to test whether a claim about a parameter is valid. In analytical chemistry, significance testing is used primarily to determine whether the difference between two values comes from determinate or random errors. The effect of a particular change in the measurement protocol, analyst, or sample itself can cause a deviation from the expected result. In the case of a suspected deviation/outlier, we need to be able to confirm mathematically...
3.3K
Central Limit Theorem01:14

Central Limit Theorem

14.3K
The central limit theorem, abbreviated as clt, is one of the most powerful and useful ideas in all of statistics. The central limit theorem for sample means says that if you repeatedly draw samples of a given size and calculate their means, and create a histogram of those means, then the resulting histogram will tend to have an approximate normal bell shape. In other words, as sample sizes increase, the distribution of means follows the normal distribution more closely.
The sample size, n, that...
14.3K

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Role of the striatum in counterfactual information seeking.

Social cognitive and affective neuroscience·2026
Same author

Introducing the Multidimensional Toolkit for the Assessment of Play in Schools (M-TAPS): a reliability study.

F1000Research·2025
Same author

The Ideological Turing Test: A Behavioral Measure of Open-Mindedness and Perspective-Taking.

Cognitive science·2025
Same author

Systematic review: Effects of cholinergic signaling on cognition in human pharmacological studies.

Neuroscience and biobehavioral reviews·2025
Same author

Evidence for control of cerebral neurovascular function by circulating platelets in healthy older adults.

The Journal of physiology·2025
Same author

Anomalous Directed Percolation on a Dynamic Network Using Rydberg Facilitation.

Physical review letters·2024

Related Experiment Video

Updated: May 21, 2025

Novel Object Recognition and Object Location Behavioral Testing in Mice on a Budget
05:57

Novel Object Recognition and Object Location Behavioral Testing in Mice on a Budget

Published on: November 20, 2018

55.2K

Sample size matters when estimating test-retest reliability of behaviour.

Brendan Williams1,2, Lily FitzGibbon3, Daniel Brady4,5

  • 1Centre for Integrative Neuroscience and Neurodynamics, University of Reading, Harry Pitt Building, Reading, UK. b.williams3@reading.ac.uk.

Behavior Research Methods
|March 22, 2025
PubMed
Summary

Intraclass correlation coefficients (ICCs) reliably measure reversal learning, but accurate variance component estimation requires larger sample sizes than commonly used. Variance decomposition is crucial for robust reliability studies.

Keywords:
Cognitive flexibilityComputational modellingReinforcement learningReliabilityReversal learningSample sizeTest retest

More Related Videos

A Within-Subject Experimental Design using an Object Location Task in Rats
09:28

A Within-Subject Experimental Design using an Object Location Task in Rats

Published on: May 6, 2021

4.4K
A Behavioral Test Battery for the Repeated Assessment of Motor Skills, Mood, and Cognition in Mice
07:18

A Behavioral Test Battery for the Repeated Assessment of Motor Skills, Mood, and Cognition in Mice

Published on: March 2, 2019

19.0K

Related Experiment Videos

Last Updated: May 21, 2025

Novel Object Recognition and Object Location Behavioral Testing in Mice on a Budget
05:57

Novel Object Recognition and Object Location Behavioral Testing in Mice on a Budget

Published on: November 20, 2018

55.2K
A Within-Subject Experimental Design using an Object Location Task in Rats
09:28

A Within-Subject Experimental Design using an Object Location Task in Rats

Published on: May 6, 2021

4.4K
A Behavioral Test Battery for the Repeated Assessment of Motor Skills, Mood, and Cognition in Mice
07:18

A Behavioral Test Battery for the Repeated Assessment of Motor Skills, Mood, and Cognition in Mice

Published on: March 2, 2019

19.0K

Area of Science:

  • Psychology
  • Neuroscience
  • Statistics

Background:

  • Intraclass correlation coefficients (ICCs) are standard for assessing test-retest reliability and between-subject differences.
  • However, ICC estimates can be influenced by within-subject variability, random errors, and measurement bias.
  • Reversal learning tasks are common assays for behavioral flexibility.

Purpose of the Study:

  • To quantify the test-retest reliability of behavioral and computational measures of reversal learning using ICCs.
  • To investigate the impact of sample size on variance component estimation and its association with ICC measures via simulation.

Main Methods:

  • Utilized data from a large online sample (N=150) for behavioral and computational reversal learning measures.
  • Performed Intraclass Correlation Coefficient (ICC) analysis for reliability quantification.
  • Conducted a simulation study to assess the effects of varying sample sizes on variance component estimation.

Main Results:

  • Behavioral and computational measures of reversal learning demonstrated reliable test-retest performance.
  • Estimating between-subject, within-subject, and error variance components required sample sizes ranging from 10 to over 300.
  • ICC estimates correlated strongly with between-subject and error variance but weakly with within-subject variance.

Conclusions:

  • Robust estimation of reliability for task performance measures necessitates sample sizes exceeding those typically employed in current reliability studies.
  • Variance decomposition is essential for comprehensive reliability studies, as ICCs alone may not fully capture within-subject variability.
  • The findings underscore the need for larger sample sizes to ensure the validity and precision of reliability estimates in behavioral research.