Related Experiment Video
Updated: Aug 5, 2026
09:16
Use of a Video Scoring Anchor for Rapid Serial Assessment of Social Communication in Toddlers
Published on: March 14, 2018
The double-pass correlation: A reliability metric for subjective judgments in developmental research
Ryoichi Watanabe1, Yusuke Moriguchi2
1Research Organization of Open Innovation and Collaboration, Ritsumeikan University, 2-150 Iwakura-cho, Ibaraki, Osaka, 567-8570, Japan.
Methodsx
|July 28, 2026
Summary
Determining reliable psychological experiment parameters for children is difficult. This study found approximately 50 repeated trials are needed for reliable results in young children, and 31 for adults, using a double-pass design.
Area of Science:
- Developmental Psychology
- Cognitive Psychology
- Psychometric Methods
Background:
- Reliable psychological experiments with young children are challenging due to subjective judgments.
- The double-pass design assesses within-participant consistency but optimal trial numbers are unclear for developmental samples.
Purpose of the Study:
- To determine the number of repeated trials needed for reliable estimation in developmental samples using a double-pass design.
- To provide empirical guidance for designing shorter, developmentally appropriate psychological tasks.
Main Methods:
- Reanalyzed data from 30 children (3-6 years) and 31 adults performing color pair similarity judgments.
- Utilized a double-pass design with identical trials presented twice to assess within-participant consistency.
- Analyzed trial numbers required for stable correlations under strict and lenient criteria.
Main Results:
- Adults required ~31 trials (strict) and ~14 trials (lenient) for stable correlations.
- Children required ~50 trials (strict) and ~32 trials (lenient).
- Children reached a correlation of .30 after ~49 repetitions; reliability increased with more trials.
Conclusions:
- Approximately 50 repeated trials are recommended for young children in double-pass designs when feasible.
- A minimum of 32 repeated trials may suffice for shortened procedures in children.
- Around 31 trials are suggested for adults to ensure reliable results.
Related Concept Videos
01:29Reliability and Validity
Reliability and validity are two important considerations that must be made with any type of data collection. Reliability refers to the ability to consistently produce a given result. In the context of psychological research, this would mean that any instruments or tools used to collect data do so in consistent, reproducible ways.
02:20Correlations
Correlation means that there is a relationship between two or more variables (such as ice cream consumption and crime), but this relationship does not necessarily imply cause and effect. When two variables are correlated, it simply means that as one variable changes, so does the other. We can measure correlation by calculating a statistic known as a correlation coefficient. A correlation coefficient is a number from -1 to +1 that indicates the strength and direction of the relationship between...
00:53Correlation and Regression
In statistics, correlation describes the degree of association between two variables. In the subfield of linear regression, correlation is mathematically expressed by the correlation coefficient, which describes the strength and direction of the relationship between two variables. The coefficient is symbolically represented by 'r' and ranges from -1 to +1. A positive value indicates a positive correlation where the two variables move in the same direction. A negative value suggests a negative...
01:12Coefficient of Correlation
The correlation coefficient, r, developed by Karl Pearson in the early 1900s, is numerical and provides a measure of strength and direction of the linear association between the independent variable x and the dependent variable y.
If you suspect a linear relationship between x and y, then r can measure how strong the linear relationship is.
What the VALUE of r tells us:
The value of r is always between –1 and +1: –1 ≤ r ≤ 1.
The size of the correlation r indicates the strength of the linear...
If you suspect a linear relationship between x and y, then r can measure how strong the linear relationship is.
What the VALUE of r tells us:
The value of r is always between –1 and +1: –1 ≤ r ≤ 1.
The size of the correlation r indicates the strength of the linear...
01:23Correlation of Experimental Data
Dimensional analysis simplifies complex physical problems and guides experimental investigations, but it does not provide complete solutions. It identifies the dimensionless groups that influence a phenomenon, but experimental data is needed to establish the specific relationships and validate theoretical predictions.
For example, a spherical particle moving through a viscous fluid experiences drag. Dimensional analysis shows that the drag force depends on the particle's diameter, velocity, and...
For example, a spherical particle moving through a viscous fluid experiences drag. Dimensional analysis shows that the drag force depends on the particle's diameter, velocity, and...
01:20Spearman's Rank Correlation Test
Spearman's rank correlation test, also known as Spearman's rho, is a nonparametric method for assessing the strength and direction of association between two variables. This test is particularly valuable when the data distribution is unknown or when the assumption of normality does not hold. Named after the English psychologist and statistician Dr. Charles Edward Spearman, it serves as the nonparametric counterpart to Pearson's correlation coefficient.
Spearman's test calculates correlation by...
Spearman's test calculates correlation by...