Intra- and inter-rater agreement in the assessment of occupational exposure to metals

Among the methods of retrospective occupational exposure assessment, expert review is considered the most accurate. Although expert review provides a more objective measure of exposure, depending on the exposure of interest it may still result in a significant degree of misclassification.

To evaluate the reliability in occupational metal exposure assessment by expert review, we analysed job history data from a case-control study of a neurological disease. First, one industrial hygienist (IH) did an initial exposure assessment of the metals copper, iron and lead, blinded to case-control status, for 608 subjects who had 3033 total jobs. We then compared exposure assessments from the original review with a second blinded review of 60 job histories (306 jobs) by the same IH (intra-rater) and of 64 job histories (361 jobs) by a different IH (inter-rater).

The per cent agreements for the intra-IH comparisons were 89.6 for copper, 87.9 for iron and 94.6 for lead, whereas the inter-IH per cent agreements were 86.4 for copper, 81.1 for iron and 76.2 for lead. Based on the assumption that reliability is related to validity, we calculated an estimate of misclassification of metal exposure by one IH. Our exposure misclassification estimates show a sizable attenuation of the odds ratio, with the expected bias similar for copper and iron when using either intra- or inter-reliability results to estimate misclassification.

Our results suggest that variation in the expert assessment of metal exposure is due mainly to the difficulties involved in transforming an occupational history into an estimate of exposure.