The Basic Science of Large Language Models in Orthopaedic Surgery

Area of Science:

• Orthopaedic Surgery
• Artificial Intelligence
• Medical Informatics

Background:

• Orthopaedic surgeons rely on various sources for continuous learning and staying updated.
• Large language models (LLMs) are increasingly used for information retrieval and medical education.
• Many surgeons are uncertain about the inner workings of AI, leading to concerns about accuracy and potential errors.

Purpose of the Study:

• To provide orthopaedic surgeons with a foundational understanding of LLMs.
• To empower surgeons to critically evaluate AI-generated content and identify potential inaccuracies.
• To guide the effective integration of AI tools into clinical practice and resident education.

Main Methods:

• Review of preclinical studies underpinning LLM development.
• Analysis of AI performance through clinical examples.
• Categorization of AI failures into retrieval and reasoning errors.

Main Results:

• LLMs can produce both accurate information and significant factual errors or fabricated citations.
• Understanding LLM mechanisms helps in identifying retrieval failures (incorrect information retrieval) and reasoning failures (logical errors in processing).
• Clinical case studies illustrate the practical implications of these AI failures.

Conclusions:

• Knowledge of LLM preclinical studies is crucial for orthopaedic surgeons.
• Surgeons can better evaluate AI tools by understanding potential failure modes.
• This knowledge facilitates the responsible adoption of AI in orthopaedics for improved practice and education.