Metacognitive planning: development and validation of an online measure

Planning is the critical first stage of metacognition. Although it has long been emphasized theoretically, it has not been the subject of much empirical study due to the lack of a valid assessment tool. Because planning is a metacognitive process, online methods that collect data during task performance would much better capture it. The present study was conducted to develop an online measure of metacognitive planning. Researchers designed a puzzle task that took the form of the popular game Sokoban, and the ratio between planning time and total time of each item was chosen as the metacognitive planning index. The task was administered to a heterogeneous sample of 440 participants composed of college students as well as 5th-, 7th-, and 10th-grade students. The results showed that valid inference could be made from the time ratio score. Cronbach's alpha and test-retest correlation provided robust evidence of reliability of the time ratio score. Confirmatory factor analysis further confirmed its unidimensionality. Validity evidence also supported the use of the time ratio score. After controlling for demographic variables, intelligence, and motivation, the time ratio score still accounted for a significant proportion of variance of Sokoban performance, the Tower of London performance, and academic achievement. The time ratio score was also found to increase with age. Taken together, the results of the study revealed that the time ratio is a psychometrically sound online measure of metacognitive planning.