Large language models in orthopedics: An exploratory research trend analysis and machine learning classification.

Area of Science:

• Orthopedic research
• Artificial Intelligence
• Machine Learning

Background:

• Large Language Models (LLMs) show significant potential to revolutionize orthopedic practice.
• A growing body of research highlights LLMs' applications in orthopedics.
• This study explores research trends and validates a machine learning classifier for orthopedic LLM publications.

Purpose of the Study:

• To analyze research trends in orthopedic Large Language Models (LLMs).
• To validate a machine learning classifier for categorizing orthopedic LLM publications into predefined domains.
• To project future growth trends in orthopedic LLM research.

Main Methods:

• Bibliometric analysis of 140 Scopus-indexed publications (2019-2024) using keyword co-occurrence and thematic clustering.
• Categorization of articles into five domains: Patient Education, Research and Ethics, Surgeon Education, Clinical Support, and Diagnostics and Radiology Interpretation.
• Machine learning classifiers (SVM) trained on TF-IDF vectorized text, evaluated using precision, recall, F1-score, and AUC-ROC.

Main Results:

• LLM publications surged from 28 in 2023 to 108 in 2024.
• The SVM model achieved 82% accuracy (AUC-ROC: 0.97), with high precision in Clinical Assistance and strong recall in Diagnosis.
• Patient Education showed balanced performance (88% precision, 78% recall), but terminology overlap caused misclassifications.

Conclusions:

• LLMs offer advancements in patient engagement and surgeon training but require careful attention to reliability and ethics.
• The validated SVM classifier is a valuable tool for navigating the expanding orthopedic LLM literature.
• Future research should focus on real-world validation and integrating multimodal AI systems.