Whose IQ is it?--Assessor bias variance in high-stakes psychological assessment

Assessor bias variance exists for a psychological measure when some appreciable portion of the score variation that is assumed to reflect examinees' individual differences (i.e., the relevant phenomena in most psychological assessments) instead reflects differences among the examiners who perform the assessment. Ordinary test reliability estimates and standard errors of measurement do not inherently encompass assessor bias variance. This article reports on the application of multilevel linear modeling to examine the presence and extent of assessor bias in the administration of the Wechsler Intelligence Scale for Children-Fourth Edition (WISC-IV) for a sample of 2,783 children evaluated by 448 regional school psychologists for high-stakes special education classification purposes. It was found that nearly all WISC-IV scores conveyed significant and nontrivial amounts of variation that had nothing to do with children's actual individual differences and that the Full Scale IQ and Verbal Comprehension Index scores evidenced quite substantial assessor bias. Implications are explored.