Executive functioning rating scales: Ecologically valid or construct invalid?

Executive functions are commonly measured using rating scales and performance tests. However, replicated evidence indicates weak/nonsignificant cross-method associations that suggest divergent rather than convergent validity. The current study is the first to investigate the relative concurrent and predictive validities of executive function tests and ratings using (a) multiple gold-standard performance tests, (b) multiple standardized rating scales completed by multiple informants, and (c) both performance-based and ratings-based assessment of academic achievement-a key functional outcome with strong theoretical links to executive function.

A well-characterized sample of 136 children oversampled for ADHD and other forms of child psychopathology associated with executive dysfunction (ages 8-13; 68% Caucasian/non-Hispanic) completed a counterbalanced series of executive function and academic tests. Parents/teachers completed executive function ratings; teachers also rated children's academic performance.

The executive function tests/ratings association was modest (r = .30) and significantly lower than the academic tests/ratings association (r = .63). Relative to ratings, executive function tests showed significantly higher cross-method predictive validity and significantly better within-method prediction; executive function ratings failed to demonstrate improved within-method prediction. Both methods uniquely predicted academic tests and ratings.

These findings replicate prior evidence that executive function tests and ratings cannot be used interchangeably as executive function measures in research and clinical applications, while suggesting that executive function tests may have superior validity for predicting academic behavior/achievement. (PsycInfo Database Record (c) 2020 APA, all rights reserved).