DeepSeek^TM and lacrimal drainage disorders: hype or is it performing better than ChatGPT^TM?

This study aimed to report the performance of the large language model DeepSeek^TM (DeepSeek TM, Hangzhou, China) and perform a head-to-head comparison with ChatGPT^TM (OpenAI, San Francisco, USA) in the context of lacrimal drainage disorders.

Questions and statements were used to construct prompts to include common and uncommon aspects of lacrimal drainage disorders. Prompts avoided covering new knowledge beyond February 2024. Prompts were presented at least twice to the latest versions of DeepSeek^TM and ChatGPT^TM [Accessed February 15-18, 2025]. A set of assessed prompts for ChatGPT^TM from 2023 (ChatGPT-2023) was utilized in this study. The responses of DeepSeek^TM and ChatGPT^TM were analyzed for evidence-based content, updated knowledge, specific responses, speed, and factual inaccuracies. The responses of the current ChatGPT^TM were also compared with those of 2023 to assess the improvement of the artificial intelligence chatbot. Three lacrimal surgeons graded the responses into three categories: correct, partially correct, and factually incorrect. They also compared the overall quality of the response between DeepSeek^TM and ChatGPT^TM based on the overall content, organization, and clarity of the answers.

25 prompts were presented to the latest versions [February 2025] of DeepSeek^TM and ChatGPT^TM. There was no significant difference in the speed of response. The agreement among the three observers was high (96%) in grading the responses. In terms of the accuracy of the responses, both AI models were similar. DeepSeek's responses were graded as correct in 60% (15/25), partially correct in 36% (9/25), and factually incorrect in 4% (1/25). ChatGPT-2025 responses were graded as correct in 56% (14/25), partially correct in 40% (10/25), and factually incorrect in 4% (1/25). Compared to 2023, ChatGPT-2025 gave responses which were more specific, more accurate, less generic with lesser recycling of phrases. When confronted with inaccuracies, both admitted and corrected the mistakes in subsequent responses. Both the AI models demonstrated the capability of challenging incorrect prompts and premises.

DeepSeek^TM was not superior but comparable to ChatGPT^TM in the context of lacrimal drainage disorders. Each had unique advantages and could complement each other. They need to be specifically trained and re-trained for individual medical subspecialties.