Related Experiment Video
Updated: Jun 3, 2026

Testing for Metacognitive Responding Using an Odor-based Delayed Match-to-Sample Test in Rats
Published on: June 18, 2018
Improvements in Cogstate Test Performance Depend on Number and Frequency of Prior Tests: Evidence from a Randomized
Audrey R Murchland1,2, Ruijia Chen2, Deborah Blacker1,3,4
1Department of Epidemiology, Harvard T.H. Chan School of Public Health.
Objectives:
Practice effects (PEs), improvements in cognitive test performance with repeated exposure, must be addressed in longitudinal studies of cognitive aging. Although many cognitive assessments are marketed as robust to PEs, evidence is limited. Randomizing testing features enables direct quantification of PEs but remains underutilized.
Methods:
Among Nurses' Health Study II participants (N=14,802), we examined PEs in the Cogstate Brief Battery arising from increased testing repetition and frequency using conditionally randomized 6- or 12-month regimens (2014 to 2019).
Results:
Taking the assessment twice previously, compared with once previously, at the 12-month assessment (defined as frequency PEs) was associated with 0.13 SD-higher global cognitive scores (95% CI: 0.10-0.16), corresponding to between-person age differences of 4.3 to 6.8 years. Taking the second assessment 6-months compared with 12-months after baseline (defined as frequency of PEs) was associated with 0.04 SD-higher global cognitive scores ( P <0.01), corresponding to between-person 1.3 to 1.6-year age differences. Repetition and frequency PEs both appeared to be greater in magnitude for participants who were older at baseline, but uncertainty was high in formal tests of effect modification.
Conclusions:
Over short follow-up periods, repetition PEs may obscure age-related cognitive decline using Cogstate. Randomizing testing features can be used to strengthen cognitive aging research.
Related Concept Videos
Cochran's Q Test
Multiple Comparison Tests
It would be easy to compare two samples using a significance alpha level of 0.05. In other words, there is only one sample pair to be compared. However, it would be difficult to identify a significantly different sample if the number...
Comparing Experimental Results: Student's t-Test
Crossover Experiments
Crossover designs are performed even with smaller sample sizes since the samples can act as their controls. These are better than simple randomized trials since patients are exposed to all the treatments.
Significance Testing: Overview
Longitudinal Research
