An assessment of inter-observer agreement and accuracy when reporting plain radiographs

This study investigates the inter-observer agreement and accuracy of plain radiograph reporting. Five consultant radiologists were asked to report 50 plain radiographs where the diagnosis had been established. Thirty-one of the radiographs showed an abnormality; 19 were normal. The radiographs were viewed on two separate occasions, 5 months apart. Initially clinical details were omitted but on the second occasion they were available. The accuracy of the report to the known diagnosis was assessed using McNemar test and inter-observer agreement was assessed using Kappa statistics. The precision (mean accuracy) improved in the light of the clinical information from 77% (range 62% to 86%) to 80% (range 78% to 84%), although this improvement did not reach statistical significance. Sensitivity improved from 80% to 83% and specificity from 72% to 76%. The Kappa value for inter-observer agreement improved from 0.31 to 0.58 for all radiographs, from 0.40 to 0.70 for the abnormal radiographs and from 0.24 to 0.43 for the normal radiographs. There was a trend to improved accuracy of diagnosis and inter-observer agreement with knowledge of clinical details.