A combined experimental-correlational approach to the construct validity of performance-based and self-report-based...

The ability to sustain attention is often measured with either objective performance indicators, like within-person RT variability, or subjective self-reports, like mind wandering propensity. A more construct valid approach, however, may be to assess the covariation in these performance and self-report measures, given that each of these is influenced by different sources of measurement error. If the correlation between performance-variability and self-report measures reflects the sustained attention construct, then task manipulations aimed at reducing the sustained attention demands of tasks should reduce the correlation between them (in addition to reducing mean levels of variability and mind wandering). The current study investigated this claim with a combined experimental-correlation approach. In two experiments (Ns ~ 1,500 each), participants completed tasks that either maximized or minimized the demand for sustained attention. Our demand manipulations successfully reduced the mean levels of sustained attention failures in both the objective and subjective measures, in both experiments. In neither experiment, however, did the covariation between these measures change as a function of the sustained attention demands of the tasks. We can therefore claim only minimal support for the construct validity of our measurement approach to sustained attention.