Evaluating the degree of concordance between employers' evaluations of work behavior

This study investigated employers' evaluations of work performed by six mentally retarded adults in a restaurant setting. The results suggested two important points. First, employers, when provided repeated observations of worker performance, rate employees fairly consistently along the production dimension (inter-rater reliability). Secondly, these same employers did not agree with themselves when provided double filmings of the same person working (intrarater reliability). Potential explanations for these findings and future areas of research are discussed. Finally, degree of concordance between trained observers and these same employers was estimated.