Large language models vs thrombosis experts: a comparative study on patient education and clinical decision-making in...

Large language models (LLMs) have demonstrated remarkable capabilities in various medical fields, yet their performance in thrombosis and hemostasis, particularly in patient education and complex clinical decision making, is unexplored.

We aimed to compare the quality of responses from LLMs vs thrombosis experts and assess clinician's ability to distinguish between them.

Three experts on thrombosis and hemostasis and 3 LLMs (Le Chat Pixtral Large, DeepSeek-R1, and ChatGPT-4.5) answered 3 patient education and 3 clinical decision-making queries. Thirty-seven physicians rated responses for adequacy (1 = very poor; 10 = excellent) and estimated their origin (1 = certainly LLM; 10 = certainly human). Mean differences were assessed via t-tests, medians via Wilcoxon tests, and correlations via Spearman test. All P values were adjusted with Bonferroni correction.

LLMs provided significantly better patient education responses than experts. Mean adequacy score differences were as follows: Le Chat Pixtral Large +1.6 (95% CI, 1.3-2.0; P < .01), DeepSeek-R1 +1.7 (95% CI, 1.3-2.1; P < .001), and ChatGPT-4.5 +1.9 (95% CI, 1.6-2.3; P < .001). In clinical decision-making, DeepSeek-R1 outperformed experts (+1.4; 95% CI, 1.1-1.8; P < .001), whereas Le Chat Pixtral Large (-0.3; 95% CI, -0.8-0.1; P = .96) and ChatGPT-4.5 (+0.5; 95% CI, 0.0-0.9; P = .18), performed comparably with experts. Evaluators could not distinguish between expert (median, 6.0; IQR, 3.0-8.0) and LLM-generated responses (median, 6.0; IQR, 4.0-8.0).

LLMs outperform experts in venous thromboembolism-related patient education and match or exceed them in clinical decision making, providing responses indistinguishable from experts. Although major barriers need to be addressed, LLMs have strong potential to support clinical management and patient education in the field of thrombosis and hemostasis.