Related Experiment Video
Updated: Aug 5, 2026
The Use of Mixed Reality in Custom-Made Revision Hip Arthroplasty: A First Case Report
Published on: August 4, 2022
Large language models as sources of patient information on robotic knee arthroplasty: a comparative evaluation
Mehmet Utku Ciftci1, Ali Can Koluman2, Ebru Aloğlu Çiftçi3
1Sultan 2. Abdülhamid Han Training and Research Hospital, Department of Orthopedics and Traumatology, 34668 Istanbul, Turkey.
Background:
Robotic-assisted total knee arthroplasty (rTKA) is increasingly used because of its surgical precision. However, inconsistent outcomes and high costs often lead patients to seek additional information from artificial intelligence (AI) tools. Large language models (LLMs) such as ChatGPT-4o, Gemini-2.5-Flash, and DeepSeek-V3 are commonly used, but their reliability and readability in orthopaedics remain unclear.
Objectives:
To compare the reliability, usefulness, quality, and readability of responses to common patient questions about rTKA generated by leading LLMs.
Methods:
Three LLMs answered 20 frequently asked patient questions (n = 20) identified through Google Trends and expert validation. Three orthopaedic specialists (n = 3) evaluated reliability, usefulness, and overall quality using validated scales, while readability was assessed with standard indices.
Results:
Inter-rater reliability was good to excellent (ICC = 0.728-0.879). Gemini-2.5-Flash achieved significantly higher reliability and usefulness scores than ChatGPT-4o and DeepSeek-V3 (all p < 0.05). ChatGPT-4o and DeepSeek-V3 produced more readable but less accurate content, revealing an inverse relationship between reliability and readability.
Conclusions:
Gemini-2.5-Flash provided the most reliable responses, highlighting the need for supervised integration of LLMs in patient education.