A comparison of scoring algorithms for the NIH Toolbox executive function tasks in a U.S. norming sample

Area of Science:

• Cognitive psychology
• Psychometrics
• Neuroscience

Background:

• Executive functions (EF) are crucial for behavioral, clinical, and educational success.
• Existing scoring methods for EF tasks often focus on speed or accuracy separately, lacking a comprehensive approach.
• There is a need for validated, norm-referenced scoring methods for EF assessments.

Purpose of the Study:

• To empirically evaluate and compare various scoring algorithms for executive function (EF) tests.
• To determine the most effective method for integrating speed and accuracy into a single, reliable score for EF tasks.
• To establish norm-referenced scores for the NIH Toolbox Version 3 (NIHTB V3) executive function measures.

Main Methods:

• Utilized data from a large, diverse U.S. population sample (N = 3,794, ages 3-85) for the NIH Toolbox Version 3 (NIHTB V3) norming study.
• Compared multiple scoring algorithms, including joint speed-accuracy integration methods (rate-correct, linear integrated, additive) against single-index scores.
• Administered and analyzed data from the Dimensional Change Card Sort and Flanker tests, two key EF measures.

Main Results:

• Joint scoring algorithms, particularly the rate-correct score, demonstrated superior psychometric properties compared to accuracy or speed alone.
• Integrated speed-accuracy scores showed consistency, stability across tasks and time, and appropriate age-related performance trajectories.
• The rate-correct score exhibited minimal ceiling effects and high sensitivity to age-related EF variability, supporting its use for norming.

Conclusions:

• The rate-correct score is recommended for scoring executive function tests due to its robust psychometric evidence and sensitivity.
• This scoring method provides a reliable basis for generating new norm-referenced scores for the NIHTB V3 EF measures.
• The findings support the use of integrated speed-accuracy scoring for improved assessment of cognitive flexibility and inhibitory control across the lifespan.