Sputum cytology within and across laboratories. A reliability study

Current research has established that diagnostic consistency in sputum cytology is a known challenge. It was already known that variability exists in diagnostic interpretations across different laboratories. However, the extent of this variability and its implications for diagnostic reliability remain unclear. No prior work had resolved how much of this variability stems from observer differences versus laboratory-specific practices. This gap motivated a closer look at inter- and intraobserver agreement in sputum cytology. Prior studies have shown that diagnostic categories can influence agreement rates. Yet, the specific impact of category definitions on observer reliability has not been fully explored. That uncertainty drove this study to assess agreement patterns using a standardized six-category system. This paper's contribution is to provide detailed agreement data across multiple laboratories and observer groups.

The study aimed to evaluate the reliability of six-category sputum cytology diagnoses across and within laboratories. It sought to determine whether observer experience or exposure to a specific preparation method affects agreement rates. The specific problem addressed is the lack of standardized benchmarks for acceptable agreement in diagnostic cytology. The motivation for this study stems from the need to improve diagnostic consistency in clinical practice. Researchers wanted to assess whether interlaboratory variability is a significant concern. They also aimed to identify which diagnostic categories show the highest and lowest agreement. This information could inform training programs or guideline revisions. The study's goal was to provide data to support decisions about diagnostic standardization.

Eleven cytotechnologists from three U.S. states were included in the reliability study. Six-category diagnoses were assigned to sputum slides prepared using the Saccomanno method. Intraobserver agreement was measured by comparing diagnoses from the same observer across multiple sessions. Interobserver agreement was calculated by comparing diagnoses from different observers within the same laboratory. Interlaboratory agreement was assessed by comparing results across all three sites. Agreement was quantified using within-1 and within-2 category metrics. Statistical significance was evaluated using kappa values. The study design allowed for repeated assessments to capture variability over time.

Intraobserver agreement within 1 category ranged from 77% to 93%. Within-2 category agreement reached 90-100% in most cases. Interobserver within-1 category agreement varied between 47% and 92%. Within-2 category agreement was higher, at 83-100%. Agreement exceeded chance levels in 69% of all observer pairings. Intralaboratory agreement was 40% in California and 40-57% in Texas. Interlaboratory agreement ranged from 13% to 60% across multiple sessions. Agreement with the Texas standard was 17-50% depending on observer and occasion.

The authors suggest that agreement is acceptable for extreme atypia categories. They propose that training or guideline refinement may improve reliability for lower categories. The study found that experience and exposure to the Saccomanno method had similar impacts on agreement. Statistical analyses may benefit from condensing the six-category scale into three to four categories. The results indicate that observer variability is a significant factor in diagnostic consistency. The authors emphasize that diagnostic reliability is achievable but requires standardization efforts. They suggest that further work is needed to define acceptable thresholds for diagnostic agreement. The findings support the need for ongoing quality assurance measures in sputum cytology.