Measuring the validity and reliability of ergonomic checklists

Ergonomic practitioners commonly use observational assessment tools, also known as checklists, to identify job hazards with regard to musculoskeletal disorders. However, it is often difficult to determine how effective such checklists are at identifying jobs in which workers are at risk, which complicates resource allocation. A means of dynamically assessing validity is needed.

This paper focuses on a simple technique with which practitioners can assess the probability that a positive checklist indication accurately identifies an at-risk job. The technique can also be used to study the effect of changes to the checklist and determine whether or not they improve the practical utility of the checklist. Similarly, by manipulating the role of different risk factors assessed on the checklist, it may guide hypotheses as to the relative importance of the risk factors. Finally, the paper briefly suggests the use of control charts to assess and manage inter- and intra-rater reliability rather than more traditional assessment methods such as correlations, Cohen's and Fleiss' kappa.

The probability that a checklist correctly identifies jobs with regard to risk of musculoskeletal injury is a useful means of assessing the checklist's validity.