The validity and reliability of diagnoses of work-related mental ill-health

To establish the reliability and validity of work-related mental ill-health diagnoses.

A UK-based surveillance scheme for work-related ill-health involving occupational physicians (OPs) reporting suggests that mental ill-health incidence is increasing by around 13% per year, with anxiety, depression and "other work-related stress" being the most common diagnoses. There have been no studies of the validity and reliability of such diagnoses. Given the existence of a large network of psychiatrists (PSYs) also involved in surveillance of work-related ill-health, an opportunity arose to measure the concurrent validity and reliability of work-related mental ill-health diagnoses.

100 anonymised summaries of cases previously reported by OPs or PSYs were collected; each was sent to 5 PSYs and 5 OPs, who assigned a diagnosis and judged whether the case was work-related. Concurrent validity of the ill-health aspect of the diagnoses, and of the opinion as to work-relatedness, was assessed by comparing the overall classifications of cases by OPs and PSYs. Reliability of the diagnostic classification was measured by kappa matrices.

Diagnostic proportions for PSYs and OPs demonstrated good agreement for anxiety, depression, anxiety plus depression and "stress" (11%, 34%, 27%, 14%) and (14%, 30%, 27%, 17%), respectively. In both groups, kappa coefficients were high for a psychotic diagnosis (0.78, 95% CI: 0.74 to 0.83), but not as high for anxiety (0.27, 95% CI: 0.23 to 0.32), depression (0.34, 95% CI: 0.29 to 0.38) and "stress" (0.15, 95% CI: 0.10 to 0.19). The odds ratio of classifying a case as work-related among PSYs compared to OPs was 2.39 (95% CI: 1.68 to 3.38), p<0.001.

The overall agreement between OPs and PSYs on mental ill-health diagnoses suggests that OP diagnoses are valid for epidemiological purposes. However, the within-group reliability of the diagnosis "stress" is low. Given differences in judgements about work-relatedness, further research is needed to investigate this aspect of a diagnosis.