Related Experiment Video
Updated: Jan 8, 2026
Author Spotlight: Revolutionizing Remote Surgery with Augmented Reality and Robotics for Enhanced Precision and Accessibility
Published on: August 9, 2024
Accuracy of artificial intelligence in carpal tunnel syndrome management: A comparative analysis of ChatGPT-4o and
Prabhjot Singh1, Lara Shqair1, Om Naphade1
1Icahn School of Medicine at Mount Sinai, Department of Orthopaedic Surgery, New York, NY, United States.
Purpose:
This study evaluated the accuracy of leading AI models, ChatGPT-4o and Gemini 1.5 Pro, in providing management recommendations for CTS in patient scenarios against American Academy of Orthopedic Surgery (AAOS) guidelines.
Methods:
Treatment ratings for CTS patient scenarios from the AAOS Appropriate Use Criteria for Management of Carpal Tunnel Syndrome Pathology were compared with ratings provided by ChatGPT and Gemini 1.5 Pro using a scale from 1 to 9, with discrepancies in treatment rating calculated by contrasting scores with AAOS ratings. Spearman correlations and paired t-tests (α < .05) were conducted to assess consensus, while heatmaps were employed to display the findings.
Results:
A total of 810 paired scores were generated across 135 patient scenarios. Compared to AAOS guidelines, ChatGPT-4o under-recommended steroid injection (mean error -2.7 ± 1.2; P < .001) and carpal tunnel release (mean error -1.8 ± 1.7; P < .001) while over-recommending electrodiagnostic studies (mean error 4.5 ± 3.8; P < .001). Gemini 1.5 Pro demonstrated a similar pattern, under-recommending steroid injection (mean error -2.4 ± 1.4; P < .001) and carpal tunnel release (mean error -1.8 ± 1.3; P < .001) but showing a less pronounced over-recommendation for electrodiagnostic studies (mean error 3.7 ± 3.2; P < .001). Gemini 1.5 Pro outperformed ChatGPT-4o in alignment with AAOS guidelines, with a stronger Spearman correlation (Rho = 0.782 vs. 0.53, P < 0.001).
Conclusions:
Gemini 1.5 Pro showed a stronger overall alignment with AAOS guidelines, indicating a more refined diagnostic approach. Ultimately, these platforms are bound by the limitations of algorithmic biases, posing a risk for misdiagnosis.
Level Of Evidence:
Level III - Non-Experimental Study.
