Inter-Reader Variability in the Classification of Pneumoconiosis Using the International Labour Organization...

Pneumoconiosis remains a major problem globally with a much higher burden in low- and middle-income countries (LMICs). There is increased inter-reader variability in the classification of pneumoconiosis in most LMICs. Despite the use of the International Labour Organization (ILO) International Classification System that standardizes reporting of pneumoconiosis on chest radiographs, inter-reader variability persists. We aimed to assess inter-reader concordance in classifying pneumoconiosis using the ILO International Classification System among artisanal and small-scale gold miners (ASGMs) in Zimbabwe. A retrospective cross-sectional study was conducted using 576 eligible chest radiographs (dicom format). Overall agreement for binary classification was moderate (κ = 0.522, 95% CI: 0.475-0.569), with pairwise agreement ranging from moderate to substantial (κ = 0.464, 95% CI: 0.389-0.538 to κ = 0.681, 95% CI: 0.602-0.761). Agreement was consistently higher for radiographs not suggestive of pneumoconiosis, with agreement ranging from 85.9% to 94.4%. Across the four ILO profusion categories, substantial inter-reader agreement was observed, with linear weighted kappa ranging from (κ = 0.554, 95% CI: 0.483-0.625 to κ = 0.668, 95% CI: 0.598-0.739) and quadratic weighted kappa values ranging from (κ = 0.675, 95% CI: 0.604-0.747 to κ = 0.755, 95% CI: 0.685-0.826). Cross-tabulation analyses demonstrated that disagreement occurred predominantly between adjacent profusion categories. Category-specific analyses showed the highest concordance for Categories 0 and 3, while Category 1 demonstrated the poorest agreement, with κ values ranging from (κ = 0.091, 95% CI: 0.004-0.178 to κ = 0.216, 95% CI: 0.067-0.365). Inter-reader agreement was slightly higher among chest radiographs of ASGMs with previous tuberculosis treatment, with κ values ranging from (κ = 0.572, 95% CI: 0.360-0.784 to κ = 0.743, 95% CI: 0.565-0.921) than among those without with κ values ranging from (κ = 0.376, 95%C1: 0.291-0.460 to κ = 0.602, 95% C1: 0.498-0.706). These findings reflect substantial variability in classifying early or borderline pneumoconiotic changes. These findings highlight variability in radiographic interpretation and demonstrate the need for enhanced training, quality control, and consensus reading approaches in ASGMs' screening programs.