Interobserver Variability Across Whole-Slide Imaging Systems

Area of Science:

• Digital Pathology
• Medical Imaging
• Artificial Intelligence in Healthcare

Background:

• Whole-slide images (WSIs) are critical for diagnostics and AI development.
• Adequate WSI quality is essential, but standards for adequacy are not well-defined.
• Inadequate slides require re-scanning, highlighting the need for clear quality benchmarks.

Purpose of the Study:

• To assess inter-cohort agreement in evaluating WSI quality and diagnostic adequacy.
• To identify variability in WSI quality assessment across different observer groups and scanners.

Main Methods:

• 180 WSIs from 60 slides scanned on three different platforms (Leica GT450, Hamamatsu NanoZoomer S360, Huron TissueScope iQ) were evaluated.
• 12 observers from four groups (attending physicians, fellows, technicians) assessed WSIs for quality (pass/fail).
• Cohen's kappa was calculated to measure interobserver agreement; WSI pass rates were compared across cohorts and scanners.

Main Results:

• Interobserver agreement was generally weak to very poor (Cohen's kappa 0.21-0.4 or <0.2) across most cohorts and scanners.
• Scanner performance rankings varied significantly depending on the observer cohort.
• Technicians favored the GT450, fellows the TissueScope iQ, and attending physicians the NanoZoomer S360.

Conclusions:

• Significant variability exists in WSI quality ratings among observers and cohorts.
• Establishing uniform, reproducible WSI quality standards is vital for patient safety, consistent care, and reliable AI model development.
• Standardization will improve the efficiency of re-scanning subpar slides and ensure AI models are trained on clinically relevant images.