Appraisal of ChatGPT's responses to common patient questions regarding acromioclavicular joint dislocations

ChatGPT, an expanding artificial intelligence platform, is rapidly becoming a source of medical knowledge for patients. The purpose of this study is to evaluate the quality and readability of information provided by ChatGPT 4.0 in response to the most frequently asked patients' questions regarding acromioclavicular (AC) joint dislocations.

Twenty-five frequently asked patient questions regarding AC joint dislocations were posed to ChatGPT 4.0. The quality and accuracy of the responses was graded by two fellowship-trained shoulder surgeons using a 5-point Likert scale. Responses were analyzed for readability using six established metrics.

ChatGPT provided responses to 25 frequently asked questions, with consensus reached on 12 questions. The final average score was 4/5, reflecting good quality with minor inaccuracies. Treatment-related questions scored lower compared to other categories, with a significant difference observed between treatment and rehabilitation (P = .025). Cohen's kappa indicated poor correlation (0.085), though the percent agreement was 48%, increasing to 100% when allowing for a 1-point difference. Readability scores revealed moderate difficulty levels, suitable for a high school-level audience.

ChatGPT delivers accurate and easily comprehensible information on AC joint dislocations, highlighting its potential to improve patient education. Although the model generally provides high-quality responses, its limitations in addressing treatment-related questions underscore the importance of clinician oversight. ChatGPT can therefore serve as a valuable complement to traditional patient education methods.