Large language models with image processing in automated Cobb angle.

Area of Science:

• Medical imaging analysis
• Artificial intelligence in healthcare
• Spinal deformity assessment

Background:

• Scoliosis severity is quantified by the Cobb angle measured on radiographs.
• The integration of artificial intelligence (AI) into clinical workflows raises questions about its utility in spinal deformity classification.
• Evaluating the capabilities of leading large language models (LLMs) in clinical applications is crucial.

Purpose of the Study:

• To assess the diagnostic accuracy of four prominent LLMs (ChatGPT, Gemini, Perplexity, Grok) in calculating Cobb angles from scoliosis radiographs.
• To compare the performance of these LLMs against a radiologist's measurements.
• To evaluate the usability and clinical applicability of AI in Cobb angle determination.

Main Methods:

• A cross-sectional study involving 122 scoliosis patient radiographs.
• Independent Cobb angle calculation by a fellowship-trained radiologist using Horos software as the reference standard.
• Radiographs were analyzed by four LLMs to identify scoliosis type, generate overlays, and calculate Cobb angles, with subsequent statistical analysis.

Main Results:

• Gemini and ChatGPT showed significant limitations, with Gemini failing to produce measurements and ChatGPT exhibiting large errors (MAE 58.6° ± 45.9°).
• Perplexity and Grok provided estimates for all cases, but with mean differences of 18.8° (±13.3°) and 24.2° (±18.3°), respectively.
• No AI model accurately identified S-shaped scoliosis or met the clinically acceptable difference (≤10%) for Cobb angle measurement.

Conclusions:

• Commercially available LLMs currently exhibit insufficient accuracy for clinical Cobb angle measurement in scoliosis.
• While Perplexity and Grok showed better performance among the tested models, none achieved clinically acceptable accuracy.
• Development of specialized AI tools is necessary for reliable clinical integration in spinal deformity assessment.