Sputum cytology within and across laboratories. A reliability study

A total of 11 cytotechnologists at sites in Texas (TX1, TX2), California (CA) and Arkansas (AR) were assessed for agreement of six-category diagnoses of sputum cytology slides prepared by the method of Saccomanno. For three observers at TX1 there was more agreement within observers (27-60%) than across observers (13-50%). Within-1 category intraobserver agreement underwent a twofold to threefold increase, to 77-93%; within-2 category agreement was 90-100%. Interobserver within-1 category agreement was 47-92%; within-2 category agreement was 83-100%. Agreement was significantly greater than chance (using kappa) in 69% of all intraobserver and interobserver pairings. Intralaboratory agreement was 40% for CA and 40-57% for TX2. Among pairings of the four sites, the range of interlaboratory agreement was 13-60% over several occasions. The overall range of agreement with the TX1 standard was 17-50% over observers/occasions. Within certain categories, outside agreement with the TX1 standard was 53-90% for normal, 39-80% for squamous metaplasia, 68-84% for mild atypia, 80-100% for moderate atypia and 93-100% for severe atypia or carcinoma. We conclude that agreement is acceptable for extreme atypia, but more training or refinement of the guidelines may be needed, if justified, to better differentiate the lowest categories. Good agreement appears to be as likely for observers with many years of overall experience as for those with high exposure to the Saccomanno method. For potential statistical analyses, the scale should probably be condensed into three to four categories to reduce extraneous variability.