Performance evaluation of an end-to-end fully automated AI-assisted gram staining system using clinical urine...

Fully automated Gram staining systems integrating staining, microscopy, image acquisition, and artificial intelligence (AI)-assisted interpretation have the potential to reduce laboratory workload and standardize workflows. However, their diagnostic performance in routine clinical practice remains unclear. We evaluated a fully automated Gram staining system using clinical urine specimens from multiple institutions.

In this prospective multicenter study, residual urine specimens from two Japanese university hospitals were evaluated using Mycrium, a fully automated system integrating Gram staining, microscopy, image acquisition, and AI-assisted interpretation. The standalone performance of the initial and updated AI models and the performance of manual microscopy were compared with a consensus reference standard established by three experienced clinical microbiology technologists.

Among 892 collected specimens, 882 were eligible for analysis. Manual microscopy demonstrated a sensitivity of 93.4%, specificity of 89.9%, and Cohen's κ of 0.83 for overall Gram stain interpretation. The initial AI model achieved a sensitivity of 91.0%, specificity of 49.3%, and κ of 0.41, whereas the updated model showed a sensitivity of 88.3%, specificity of 59.0%, and κ of 0.48. Model updating resulted in category-specific trade-offs between sensitivity and specificity. Most false-negative results occurred in 1+ specimens.

The automated Gram staining system demonstrated high sensitivity but moderate specificity. Although its diagnostic performance remained inferior to experienced technologists, it enabled a fully automated workflow integrating staining, microscopy, image acquisition, and AI-assisted interpretation. These findings support the feasibility of integrated Gram stain workflow automation and provide a benchmark for further refinement of automated microscopy and AI algorithms.