Evaluation of interrater reliability for posture observations in a field study

Area of Science:

• Occupational health and safety
• Ergonomics
• Biomechanics

Background:

• Assessing non-neutral postures in the workplace is crucial for identifying ergonomic risks.
• Quantitative observational methods are used to evaluate physical exposures during work tasks.
• Understanding the reliability of these assessment methods is essential for accurate risk evaluation.

Purpose of the Study:

• To evaluate the interrater reliability of a quantitative observational method for assessing non-neutral work postures.
• To compare different statistical methods for assessing interrater reliability in ergonomic assessments.
• To identify potential sources of variability in reliability and validity studies.

Main Methods:

• Two independent observers evaluated 70 jobs in an automotive manufacturing facility.
• The observational method focused on 18 postures of the upper extremities and back.
• Interrater reliability was assessed using percent agreement, kappa, intraclass correlation coefficients, and generalized linear mixed modeling.

Main Results:

• Interrater agreement varied significantly across different postures, from 26% for right shoulder elevation to 99% for left wrist flexion.
• Kappa statistics indicated only moderate agreement for most posture assessments.
• Percent agreement was found to be an inadequate reliability measure due to its failure to account for chance agreement.

Conclusions:

• Interrater reliability for some postural observations in this study was acceptable.
• More appropriate statistical methods, such as generalized linear mixed modeling, offer greater insight into reliability and validity.
• Improved assessment methods are needed for more effective ergonomic exposure evaluation.