Artificial Intelligence Large Language Models Address Anterior Cruciate Ligament Reconstruction: Superior Clarity and Completeness by Gemini Compared With ChatGPT-4 in Response to American Academy of Orthopaedic Surgeons Clinical Practice Guidelines
View abstract on PubMed
Summary
This summary is machine-generated.Gemini and ChatGPT-4 show good accuracy in responding to anterior cruciate ligament reconstruction guidelines. Gemini performed better in clarity and completeness, especially for rehabilitation and prevention information.
Area Of Science
- Orthopaedic surgery
- Artificial intelligence
- Clinical practice guidelines
Background
- Large language models (LLMs) are increasingly used for medical information.
- The 2022 American Academy of Orthopaedic Surgeons (AAOS) Clinical Practice Guidelines (CPG) for anterior cruciate ligament reconstruction (ACLR) provide evidence-based recommendations.
- Evaluating the accuracy and relevance of LLM responses to CPGs is crucial for patient education and clinical decision-making.
Purpose Of The Study
- To compare the performance of ChatGPT-4 and Gemini in generating accurate and relevant responses to the 2022 AAOS CPG for ACLR.
- To assess the clarity and completeness of LLM-generated answers based on expert surgeon evaluation.
Main Methods
- Seven orthopaedic sports medicine surgeons evaluated responses from ChatGPT-4 and Gemini to prompts derived from all 15 AAOS ACLR guidelines.
- Responses were assessed using a structured questionnaire on 5 key characteristics (scale 1-5).
- Prompts covered diagnosis, surgical timing, and rehabilitation; statistical analysis compared LLM performance and inter-rater reliability.
Main Results
- Both LLMs achieved high overall mean scores (>4).
- Gemini demonstrated statistically superior overall clarity (P=.034) and in surgical timing/technique (P=.038) and rehabilitation/prevention (P=.044) subcategories.
- Gemini also showed superior completeness in rehabilitation and prevention (P=.044).
Conclusions
- Both ChatGPT-4 and Gemini can generate accurate and relevant responses to ACLR CPG questions.
- Gemini exhibits superior clarity across multiple domains and superior completeness in rehabilitation and prevention.
- These findings highlight Gemini's potential as a more effective tool for patient education regarding ACLR.