Evaluation of interrater reliability for posture observations in a field study

This paper examines the interrater reliability of a quantitative observational method of assessing non-neutral postures required by work tasks. Two observers independently evaluated 70 jobs in an automotive manufacturing facility, using a procedure that included observations of 18 postures of the upper extremities and back. Interrater reliability was evaluated using percent agreement, kappa, intraclass correlation coefficients and generalized linear mixed modeling. Interrater agreement ranged from 26% for right shoulder elevation to 99 for left wrist flexion, but agreement was at best moderate when using kappa. Percent agreement is an inadequate measure, because it does not account for chance, and can lead to inflated measures of reliability. The use of more appropriate statistical methods may lead to greater insight into sources of variability in reliability and validity studies and may help to develop more effective ergonomic exposure assessment methods. Interrater reliability was acceptable for some of the postural observations in this study.