Performance of 3 large language models in detecting urinary formed elements

The integration of large language models (LLMs) into medical laboratories has gained significant attention, especially in the field of image recognition. However, research on their ability to identify unstained images, such as those of urinary formed elements, remains scarce. This study assesses the image recognition capabilities of 3 LLMs, ChatGPT-4o, Deepseek Janus Pro7B and Google Gemini, in detecting urinary formed elements. This cross-sectional study analyzed 45 urine morphology images, utilizing a standardized prompt to guide ChatGPT-4o, Deepseek Janus Pro7B and Google Gemini in recognizing urine formed elements. Each image was independently evaluated 3 times, and the results were assessed using a 5-point Likert scale. The accuracy and consistency of the models were compared through the Friedman test, Kendall W score, and the Mann-Whitney U test. Gemini Advanced demonstrated superior performance with a 31% accuracy rate. ChatGPT-4o (W = 0.797) and Gemini (W = 0.812) exhibited strong consistency, suggesting a superior performance compared to Deepseek (W = 0.663). Statistical analysis revealed a significant difference in performance between the 3 models, with Gemini showing superior ability to identify urinary formed elements, particularly cast and microorganism. While ChatGPT-4o, Deepseek Janus Pro7B and Google Gemini show promise and exhibit potential in identifying urinary formed elements, their diagnostic performance remains limited and are currently inadequate for clinical use in identifying urine morphology. To enhance their clinical applicability, further improvements in training data and model optimization are required. Future research should focus on enhancing these models' performance to ensure their broader utility in medical laboratories.