Executive functioning rating scales: Ecologically valid or construct invalid?

Area of Science:

• Cognitive Psychology
• Neuropsychology
• Developmental Psychology

Background:

• Executive functions are crucial for academic achievement.
• Commonly, executive functions are assessed via performance tests and rating scales.
• Prior research shows weak associations between these two measurement methods, questioning their validity.

Purpose of the Study:

• To investigate the concurrent and predictive validity of executive function tests versus rating scales.
• To compare the utility of performance-based and rating-based assessments of executive functions in predicting academic achievement.
• To examine these relationships using multiple gold-standard tests, standardized rating scales, and multiple informants.

Main Methods:

• 136 children (ages 8-13) with or at risk for psychopathology completed executive function and academic performance tests.
• Parents and teachers completed standardized rating scales for executive functions and academic performance.
• Concurrent and predictive validity were assessed by comparing test scores and ratings with academic outcomes.

Main Results:

• The association between executive function tests and ratings was modest (r = .30), significantly lower than that for academic measures (r = .63).
• Executive function tests demonstrated superior cross-method predictive validity and within-method prediction compared to rating scales.
• Both executive function tests and ratings uniquely predicted academic performance, but tests showed greater predictive power.

Conclusions:

• Executive function tests and rating scales are not interchangeable measures in research or clinical settings.
• Performance-based executive function tests appear to have superior validity for predicting academic behavior and achievement.
• Findings support the use of performance tests for a more accurate assessment of executive functions' impact on academics.