The double-pass correlation: A reliability metric for subjective judgments in developmental research

Ensuring reliable psychological experiments with young children is challenging, especially when outcomes depend on subjective judgments that vary across individuals and tasks. The double-pass design, in which identical trials are presented twice to assess within-participant consistency, is useful for evaluating reliability. However, the number of repeated trials required for reliable estimation in developmental samples remains unclear. We reanalyzed the data from 30 children aged 3-6 years and 31 adults who completed two blocks of color pair similarity judgments. In the observed order analysis, adults reached stable correlations with approximately 31 trials under a strict criterion and 14 trials under a lenient criterion, whereas children required approximately 50 and 32 trials, respectively. Children reached ρ = .30 after approximately 49 repetitions, whereas stronger thresholds were less consistent: 83.3% exceeded ρ = .50 and 63.3% exceeded ρ = .70 in the full 81-trial estimates. These findings suggest that approximately 50 repeated trials may be appropriate for young children when feasible, whereas at least 32 repeated trials may be needed as a minimum shortened procedure to obtain reliable results. For adults, approximately 31 trials may suffice. These results provide empirical guidance for designing shorter, developmentally appropriate tasks that maintain reliable interpretability.