Analysis of workload predictions generated by multiple resource theory

During the design process, operator workload assessments sometimes require subject matter experts (SME's) to predict the amount of physical and mental demands they expect during system employment. Often, these subjective estimates are considered within the context of models that partition human capabilities into discrete resources. Such models require the SME's to rate tasks on how they affect resource consumption. This type of assessment technique is based upon the premise that raters can effectively discriminate their own resources.

Subjective workload ratings based on multiple resource theory were collected independently from two highly experienced pilots for 225 different tasks of an anticipated mission for a future advanced strike aircraft. The data were examined using a factor analytic approach.

Factor analysis of their responses suggest that while such ratings have high face validity and even high inter-rater reliabilities, the ratings could have little validity in terms of efforts required to use the seven postulated resource channels (i.e., visual or auditory input, spatial, verbal or analytical cognition, and manual or speech output). Ratings of efforts required for the cognitive resource channels were particularly suspect. We identified four independent factors for each pilot that accounted for virtually all of the intercorrelations among the 7 resource channels. Three factors (i.e., visual-spatial, verbal communications, and manual and speech output) were identical for both pilots and accounted for most of the explainable variance.

Given these results, this analysis challenges the utility of a multiple resource framework for predictive workload assessments.