Large Language Model Predicts Surgeon Recommendations for Imaging and Surgery for Patients Presenting for Knee and...

Area of Science:

• Orthopaedic Surgery
• Artificial Intelligence in Medicine
• Clinical Decision Support

Background:

• Previsit questionnaires are crucial for initial patient assessment in orthopaedic sports medicine.
• Large language models (LLMs) show potential for analyzing clinical text data.
• Validating LLM performance in predicting surgical and imaging needs is essential.

Purpose of the Study:

• To evaluate a pretrained LLM's accuracy in predicting orthopaedic surgeon recommendations.
• To assess LLM performance using free-text previsit questionnaire responses.
• To determine if augmenting LLM with imaging reports improves surgical recommendations.

Main Methods:

• Retrospective analysis of 1141 new orthopaedic sports medicine patients (2020-2023).
• Zero-shot prompting LLM to predict imaging/surgery needs from questionnaire data.
• Comparison of LLM predictions against surgeon-generated plans (accuracy, sensitivity, specificity).
• LLM analysis of radiology reports for updated surgical recommendations in a subset of patients.

Main Results:

• LLM predicted imaging needs with 70% accuracy, 83% sensitivity, 64% specificity from questionnaires.
• High accuracy for ACL, meniscus, rotator cuff injuries; lower for arthritis.
• Augmented LLM predicted surgical needs with 81% accuracy, 88% sensitivity, 72% specificity.
• Excellent surgical prediction accuracy for ACL, meniscus, rotator cuff, and shoulder instability.

Conclusions:

• Pretrained LLM accurately predicts imaging recommendations (70% accuracy) from patient questionnaires.
• Augmented LLM achieves 81% accuracy in predicting surgical recommendations.
• LLMs show promise as tools for orthopaedic clinical decision support.