From Gram Stain to Decision Support: Performance of Multimodal Large Language Models in Blood Culture Microscopy

Area of Science:

• Medical Microbiology
• Artificial Intelligence in Diagnostics
• Computational Pathology

Background:

• Multimodal large language models (LLMs) present potential for image-based diagnostics.
• Reliability of LLMs in routine clinical microbiology, specifically Gram stain interpretation, is not well-established.
• Gram staining remains a cornerstone of bacterial identification in clinical laboratories.

Purpose of the Study:

• To evaluate the diagnostic accuracy of three leading multimodal LLMs (ChatGPT-4o, Gemini 2.5 Flash, Claude Opus) in interpreting Gram-stained blood culture smears.
• To benchmark LLM performance against expert medical microbiologists and automated identification systems.
• To assess LLM consistency and identify areas of strength and limitation in Gram stain interpretation.

Main Methods:

• Prospective diagnostic accuracy study involving 100 Gram-stained blood culture smear images.
• Evaluation of LLMs using standardized zero-shot prompts across two independent runs for intramodel consistency.
• Benchmarking against three expert medical microbiologists and automated organism identification as the reference standard.

Main Results:

• Expert microbiologists achieved 100% concordance with the reference standard for Gram type and major morphology.
• LLMs demonstrated high accuracy for Gram-type classification (95-98%) but lower accuracy for cellular morphology (84-85%), resulting in combined accuracies of 82-84%.
• Performance varied by organism class, with lower accuracy for Gram-positive bacilli and yeast; ChatGPT and Gemini outperformed Claude on fine-grained morphology.

Conclusions:

• Multimodal LLMs exhibit promising baseline performance for Gram-stained blood culture interpretation, particularly for Gram type and major morphology.
• Current limitations include reduced accuracy in combined interpretation, fine-grained morphology, and less common organism classes, indicating out-of-the-box models are insufficient for standalone use.
• Findings support the use of LLMs as expert-supervised decision-support tools, emphasizing the need for task-specific optimization and multicenter validation prior to clinical implementation.