Examining the Role of Large Language Models in Orthopedics: Systematic Review

Area of Science:

• Orthopedic medicine
• Artificial Intelligence
• Medical Informatics

Background:

• Large language models (LLMs) offer significant potential in medical applications, including orthopedics.
• Orthopedic diseases represent a substantial socioeconomic burden, highlighting the need for innovative solutions.
• Existing research on LLMs in orthopedics is fragmented, necessitating a systematic review.

Approach:

• A comprehensive literature search was conducted across PubMed, Embase, and Cochrane Library databases (2014-2024).
• Studies were selected based on predefined inclusion/exclusion criteria, with quality assessed using revised Cochrane risk-of-bias and CONSORT-AI tools.
• Data extraction and synthesis were performed following quality assessment.

Key Points:

• 68 studies were analyzed, focusing on clinical practice (69%), education (18%), research (12%), and management (1%).
• LLM applications varied, with ChatGPT being the most cited tool; however, significant heterogeneity existed in performance evaluation.
• Diagnostic accuracy ranged from 55-93%, disease classification from 2-100%, and examination question accuracy from 45-73.6%.

Conclusions:

• LLMs are unlikely to replace orthopedic professionals in the near future.
• LLMs can serve as valuable copilots to enhance current work efficiency in orthopedics.
• Future high-quality clinical trials are essential to define optimal LLM applications and improve orthopedic precision and efficiency.