Related Experiment Video
Updated: Jul 14, 2026
08:30
Operant Procedures for Assessing Behavioral Flexibility in Rats
Published on: February 15, 2015
Repeatedly unreliable, and unreliably repeatable: the trials-to-criterion fallacy
1Centre for Evolutionary Biology, School of Biological Sciences, The University of Western Australia, Crawley, WA, 6009, Australia. robert.dugand@uwa.edu.au.
Animal Cognition
|July 13, 2026
Summary
Trials-to-criterion (TTC) may inaccurately estimate intelligence due to chance. Simulations show that lucky streaks and occasional errors in TTC tasks can inflate scores, making results unreliable for assessing learning speed or cognitive ability.
Area of Science:
- Cognitive Science
- Psychometrics
- Behavioral Neuroscience
Background:
- Trials-to-criterion (TTC) is widely used to measure learning speed and estimate intelligence.
- TTC involves tasks with a 50% chance of success per trial, requiring a set criterion (e.g., consecutive passes) to be met.
Purpose of the Study:
- To investigate the reliability of TTC as a measure of intelligence.
- To determine the extent to which chance and variability influence TTC outcomes.
Main Methods:
- The study employed computer simulations to model TTC task performance.
- Simulations analyzed the impact of random chance, lucky streaks, and occasional errors on TTC scores.
Main Results:
- Simulations demonstrated that random chance and lucky streaks can lead to criterion attainment, conflating luck with intelligence.
- Occasional errors by intelligent individuals can significantly inflate TTC scores, misrepresenting cognitive ability.
- High variability in TTC outcomes for a fixed intelligence level highlights its unreliability.
Conclusions:
- Inferences based on TTC data may be misleading due to inherent unreliability.
- The study cautions against the use of TTC for accurately estimating intelligence or learning speed.
- Recommendations are provided for future research to mitigate artefactual results in learning assessments.
Related Concept Videos
01:53Cause and Effect
While variables are sometimes correlated because one does cause the other, it could also be that some other factor, a confounding variable, is actually causing the systematic movement in our variables of interest. For instance, as sales in ice cream increase, so does the overall rate of crime. Is it possible that indulging in your favorite flavor of ice cream could send you on a crime spree? Or, after committing crime do you think you might decide to treat yourself to a cone?
01:24Testing a Claim about Population Proportion
A complete procedure for testing a claim about a population proportion is provided here.
There are two methods of testing a claim about a population proportion: (1) Using the sample proportion from the data where a binomial distribution is approximated to the normal distribution and (2) Using the binomial probabilities calculated from the data.
The first method uses normal distribution as an approximation to the binomial distribution. The requirements are as follows: sample size is large...
There are two methods of testing a claim about a population proportion: (1) Using the sample proportion from the data where a binomial distribution is approximated to the normal distribution and (2) Using the binomial probabilities calculated from the data.
The first method uses normal distribution as an approximation to the binomial distribution. The requirements are as follows: sample size is large...
01:29Reliability and Validity
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
01:29Theory of Attribution II: Kelley's Covariation Theory
Attribution theory plays a crucial role in social psychology, helping to explain how individuals interpret the causes of behavior. One prominent model within this field is Harold Kelley's covariation theory, which provides a systematic approach to determining whether internal traits or external circumstances drive a person's actions. The model posits that individuals rely on three key types of information—consensus, consistency, and distinctiveness—to make these judgments.Consensus: Comparing...
03:37Uncertainty in Measurement: Accuracy and Precision
Scientists typically make repeated measurements of a quantity to ensure the quality of their findings and to evaluate both the precision and the accuracy of their results. Measurements are said to be precise if they yield very similar results when repeated in the same manner. A measurement is considered accurate if it yields a result that is very close to the true or the accepted value. Precise values agree with each other; accurate values agree with a true value.
01:20Random and Systematic Errors
Scientists always try their best to record measurements with the utmost accuracy and precision. However, sometimes errors do occur. These errors can be random or systematic. Random errors are observed due to the inconsistency or fluctuation in the measurement process, or variations in the quantity itself that is being measured. Such errors fluctuate from being greater than or less than the true value in repeated measurements. Consider a scientist measuring the length of an earthworm using a...