The Performance of ChatGPT-4.0 and ChatGPT-4omni on Answering Thyroid Question: A Multicenter Study

Although ChatGPT-4.0 exhibits increasing potential in medical applications, its more recent version, ChatGPT-4omni, has not yet been evaluated for how well it responds to patient questions on thyroid health. In this study, the performance of ChatGPT-4.0 and ChatGPT-4omni in answering questions on the thyroid was examined.

To test the performance of ChatGPT-4.0 and ChatGPT-4omni, we first obtained 28 thyroid-related questions from the Huayitong app, a convenient medical app that was officially released by West China Hospital of Sichuan University. We also added two interventional questions to the total of 30 questions. On June 28, 2024, we entered these queries into ChatGPT-4.0 and ChatGPT-4omni in Chinese to generate 60 Chinese replies. Finally, from July 1 to 15, 2024, we asked 60 patients, 29 surgeons, and 37 nurses from 21 tertiary care units nationwide to rate the two sources' responses on a 5-point Likert scale in terms of time, word count, response speed, accuracy, comprehensiveness, empathy, and satisfaction.

When answering 30 questions, ChatGPT-4omni answered more words (437.30 [110.20] characters versus 750.50 [611.50-817.25] characters; P < 0.001), took less time to respond (27.58 [7.22] seconds versus 20.68 [4.38] seconds; P < 0.001), and was faster (15.69 [13.90-16.92]) character/second versus 34.26 [5.03] character/second; P < 0.001) than ChatGPT-4.0. Responses from ChatGPT-4omni were rated as more accurate, comprehensive, sympathetic, and satisfied than those from ChatGPT-4.0 by patients, thyroid surgeons, and thyroid surgery nurses (all P values < 0.05).

ChatGPT-4omni outperformed ChatGPT-4.0 in answering common thyroid-related questions. However, further study and optimization are needed to achieve an efficient integration of ChatGPT in clinical settings.