Development and validation of the visual reasoning test for the NIH Toolbox

Area of Science:

• Neuroscience
• Psychology
• Cognitive Science

Background:

• The NIH Toolbox (NIHTB) is a validated iPad-based cognitive assessment battery for ages 3-85+.
• A measurement gap in visual reasoning (VR) was identified in prior NIHTB versions.
• Existing cognitive assessments may lack brief, reliable measures for VR across the full age spectrum.

Purpose of the Study:

• To develop and validate a novel computer-adaptive visual reasoning (VR) test for the NIH Toolbox (NIHTB).
• To address the identified measurement gap in VR assessment for individuals aged 3 to 85+.
• To establish the psychometric properties, including reliability and validity, of the new VR test.

Main Methods:

• Developed a new VR test using computer-adaptive testing (CAT) methodology.
• Calibrated 180 VR items on 3,768 participants in the NIHTB Version 3 norming study.
• Evaluated convergent validity using Wechsler Intelligence Scale subtests (Matrix Reasoning) in 283 participants.
• Assessed test-retest reliability in 190 participants over 1-14 days.

Main Results:

• The VR test demonstrated strong correlations with Wechsler Matrix Reasoning (r = .45–.55) and Full-Scale IQ scores (r = .48–.65).
• Test-retest reliability for VR scores was robust, with an intraclass correlation coefficient of .77.
• The calibrated item bank effectively measures VR ability across a wide spectrum.

Conclusions:

• The new NIHTB VR Test is a reliable and valid computer-adaptive measure for assessing visual reasoning from preschool through adulthood.
• This VR assessment effectively bridges a critical gap in the NIHTB, enhancing its scope.
• The test shows strong convergent validity and test-retest reliability, contributing to general cognitive ability assessment.