Performance of ChatGPT-5 in Diagnosing Fractures on Proximal Humerus and Intertrochanteric Femur X-Rays

Large language models (LLMs) such as ChatGPT-5 offer new possibilities for interpreting medical images, but their effectiveness in orthopedic radiograph analysis remains largely unexplored.

To evaluate the diagnostic performance of ChatGPT-5 in detecting and classifying fractures on shoulder and hip X-rays, specifically proximal humerus and intertrochanteric (IT) femur fractures.

A retrospective study of 120 anonymized anteroposterior (AP) radiographs (60 shoulder and 60 hip) was conducted. Each case was independently reviewed by orthopedic experts, establishing a reference standard. ChatGPT-5 analyzed the same images using structured prompts and was assessed for fracture detection accuracy, sensitivity, specificity, and agreement on detailed fracture features.

ChatGPT-5 achieved 87.5% sensitivity and 100% specificity in detecting proximal humerus fractures (κ = 0.74), and 100% sensitivity but only 16.7% specificity in IT femur fractures (κ = 0.24). While it identified major fracture patterns and comminution reliably, it frequently hallucinated fractures in normal hip X-rays and missed fine details such as lesser tuberosity fragments and dislocations.

ChatGPT-5 shows high sensitivity for orthopedic fracture detection and produces coherent, structured reports. However, limitations in specificity and fine-detail recognition restrict its autonomous clinical use. It may serve as a triage or educational tool with human oversight or be integrated into hybrid artificial intelligence workflows.