Related Experiment Video
Updated: May 14, 2026
Multimodal Protocol for Assessing Metacognition and Self-Regulation in Adults with Learning Difficulties
Published on: September 27, 2020
Reliability of remote self-administered Web-based digital cognitive measures and comparison to in-person
Morgan A Hughes1, Ryan D Frank2, Rita L Taylor3
1Department of Anesthesia and Perioperative Medicine Mayo Clinic Rochester Minnesota USA.
Introduction:
We describe the reliability of remote self-administered digital cognitive measures completed via the Mayo Test Drive (MTD) web-based platform.
Methods:
A total of 1846 participants (mean age = 70, SD = 12, range 31-101; 48% male; 96% White; 99% non-Hispanic; 97% cognitively unimpaired) with two to four complete MTD sessions at ∼7.5-month intervals were included. Test-retest reliability was assessed using single-rating, absolute-agreement, and two-way mixed intraclass correlation coefficients (ICCs) with 95% confidence intervals. ICCs for in-person-administered traditional neuropsychological measures were compared to MTD for a subset of 244 participants. Analyses also compared reliability across demographic, residential, and session context subgroups.
Results:
Reliability was good for the MTD Composite (total ICC = 0.79 [0.77, 0.80]), and moderate-to-good for the primary outcome variables for each MTD subtest (total ICCs 0.70-0.83 for Stricker Learning Span and Symbols). The reliability of the remote self-administered MTD was similar to in-person-administered cognitive measures.
Discussion:
MTD showed moderate-to-good reliability, supporting its use in longitudinal monitoring.
