Artificial intelligence models cannot yet replace experts in providing patient education for shoulder and elbow...

AI and machine learning (ML) have diverse applications in orthopedic surgery, such as for diagnosis of disease, surgical assistance, and outcome prediction. When used as adjuncts, AI has a potential to reduce clinical workload, improve workflow and aid in clinical decision making. The objective of this systematic review is to evaluate current literature on artificial intelligence (AI) to assess effectiveness in developing responses to inquiries related to orthopaedic upper extremity pathologies.

Three databases (PubMed, MEDLINE, EMBASE) were searched for studies involving AI and questions in shoulder and elbow orthopedics. Inclusion criteria included papers related to shoulder and elbow, human studies, use of AI models, published in English language and at any level of evidence. Data on response accuracy, reliability and quality, as well as area under the curve (AUC) of the given AI algorithm, were recorded. Meta-analyses were conducted on both the accuracy and AUC of AI algorithms on relevant studies. Risk of bias was assessed using the Quality Assessment of Diagnostic Accuracy Studies (QUADAS-2).

A total of 16 studies were included in this review. Nine studies used a version of ChatGPT, one study used GoogleBard, and the remaining seven studies used a variety of AI learning models. The overall pooled accuracy of responses developed by AI models was 78%, and the pooled mean AUC of included AI algorithms was 86%. AI algorithms performed inferiorly compared to experts. The overall quality and readability of AI responses were poor.

AI algorithms assessed in our study demonstrated a promising degree of accuracy and performance. However, AI responses were found to be inferior to experts and had poor readability, quality, and value to the patient. In its current state of technology, AI is a powerful tool that can be used in conjunction with experts to augment patient education, however, it should not be utilized independently.