Performance evaluation and comparison of ChatGPT, Gemini, Grok, and DeepSeek in the interpretation of tumor marker...

Large language models (LLMs) can interpret laboratory reports through prompts, but their performance remains uncertain. This study aims to evaluate and compare the interpretative abilities of different LLMs for tumor marker reports, providing a reference for their potential applications in medical laboratories.

ChatGPT-4o, Gemini-2.5, Grok-3, and DeepSeek-R1 were used to interpret 200 tumor marker reports. Two junior and two senior laboratory technologists assessed the outputs using a 5-point Likert scale. Differences among the LLMs were analyzed using the Friedman and Wilcoxon signed-rank tests.

All LLMs generated interpretable outputs. The mean scores (±SD) were: DeepSeek-R1, 4.31 ± 0.61; ChatGPT-4o, 4.25 ± 0.64; Gemini-2.5, 4.16 ± 0.75; and Grok-3, 3.82 ± 1.06. The rates of low-quality scores (≤2) and hallucinations were: ChatGPT-4o (0.1%, 1.0%), DeepSeek-R1 (0.3%, 3.0%), Gemini-2.5 (2.5%, 4.0%), and Grok-3 (10.1%, 11.5%). Score differences among the four LLMs were statistically significant (Friedman test, P < 0.001), with pairwise comparisons also showing significant differences (Wilcoxon test, all P < 0.001). Junior evaluators assigned higher scores (4.30 ± 0.50) than senior evaluators (3.97 ± 1.00). DeepSeek-R1, ChatGPT-4o, and Gemini-2.5 outperformed Grok-3, particularly in comprehensiveness, accuracy, and relevance. All LLMs performed well in identifying abnormal values but were less proficient in cause analysis and providing clinical recommendations.

Assuming LLMs can be deployed locally and sufficient computing resources are available, LLMs can serve as auxiliary tools for laboratory report interpretation, but performance varies significantly among models. Strict professional oversight and continuous improvement are necessary to minimize the risk of misinterpretation.