Assessing the knower-level framework: How reliable is the Give-a-Number task?

The Give-a-Number task has become a gold standard of children's number word comprehension in developmental psychology. Recently, researchers have begun to use the task as a predictor of other developmental milestones. This raises the question of how reliable the task is, since test-retest reliability of any measure places an upper bound on the size of reliable correlations that can be found between it and other measures. In Experiment 1, we presented 81 2- to 5-year-old children with Wynn (1992) titrated version of the Give-a-Number task twice within a single session. We found that the reliability of this version of the task was high overall, but varied importantly across different assigned knower levels, and was very low for some knower levels. In Experiment 2, we assessed the test-retest reliability of the non-titrated version of the Give-a-Number task with another group of 81 children and found a similar pattern of results. Finally, in Experiment 3, we asked whether the two versions of Give-a-Number generated different knower levels within-subjects, by testing 75 children with both tasks. Also, we asked how both tasks relate to another commonly used test of number knowledge, the "What's-On-This-Card" task. We found that overall, the titrated and non-titrated versions of Give-a-Number yielded similar knower levels, though the non-titrated version was slightly more conservative than the titrated version, which produced modestly higher knower levels. Neither was more closely related to "What's-On-This-Card" than the other. We discuss the theoretical and practical implications of these results.