Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Traumatic Brain Injury l: Introduction01:28

Traumatic Brain Injury l: Introduction

DefinitionTraumatic brain injury, or TBI, is a disturbance of normal brain function induced by an external mechanical force, such as a direct blow to the head or a penetrating injury. It can affect both brain structure and function, producing a wide range of clinical outcomes. TBI is a heterogeneous condition, meaning its effects may differ based on the type, location, and severity of the injury.Basis of ClassificationTBI is classified based on severity, injury mechanism, or pathophysiology. In...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Similar Outcomes but Significantly Different Donor Site Morbidity Profiles After Autograft Quadriceps and Bone-Patellar Tendon-Bone Anterior Cruciate Ligament Reconstruction.

Arthroscopy : the journal of arthroscopic & related surgery : official publication of the Arthroscopy Association of North America and the International Arthroscopy Association·2026
Same author

Secondary Expert Medical Opinions Associated with Treatment Strategy Changes and Substantial Cost Savings for Healthcare Payers.

Journal of healthcare management / American College of Healthcare Executives·2026
Same author

Artificial intelligence and machine learning is successful in predicting clinical outcomes after hip arthroscopy for femoroacetabular impingement syndrome.

Knee surgery, sports traumatology, arthroscopy : official journal of the ESSKA·2025
Same author

The State of Telemedicine, Telerehabilitation, and Virtual Care in Musculoskeletal Health: A Narrative Review.

HSS journal : the musculoskeletal journal of Hospital for Special Surgery·2025
Same author

Pharmacologic Mobilization and Chemokine-Directed Recruitment of Mesenchymal Stromal Cells to the Surgically Repaired Rotator Cuff.

The American journal of sports medicine·2025
Same author

Large Language Model Use Cases in Health Care Research Are Redundant and Often Lack Appropriate Methodological Conduct: A Scoping Review and Call for Improved Practices.

Arthroscopy : the journal of arthroscopic & related surgery : official publication of the Arthroscopy Association of North America and the International Arthroscopy Association·2025

Related Experiment Video

Updated: May 14, 2026

3D Ultrasound Imaging: Fast and Cost-effective Morphometry of Musculoskeletal Tissue
08:52

3D Ultrasound Imaging: Fast and Cost-effective Morphometry of Musculoskeletal Tissue

Published on: November 27, 2017

23.2K

Currently Available Large Language Models Do Not Provide Musculoskeletal Treatment Recommendations That Are

Benedict U Nwachukwu1, Nathan H Varady1, Answorth A Allen1

  • 1Department of Orthopaedic Surgery, Hospital for Special Surgery, New York, New York, U.S.A.

Arthroscopy : the Journal of Arthroscopic & Related Surgery : Official Publication of the Arthroscopy Association of North America and the International Arthroscopy Association
|August 22, 2024
PubMed
Summary

Leading large language models (LLMs) often provide treatment recommendations for rotator cuff and ACL injuries that do not align with clinical practice guidelines. Further research is needed to ensure LLM trustworthiness in healthcare.

More Related Videos

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

525
Biomechanical Changes Related to Low Back Pain: An Innovative Tool for Movement Pattern Assessment and Treatment Evaluation in Rehabilitation
06:28

Biomechanical Changes Related to Low Back Pain: An Innovative Tool for Movement Pattern Assessment and Treatment Evaluation in Rehabilitation

Published on: December 13, 2024

452

Related Experiment Videos

Last Updated: May 14, 2026

3D Ultrasound Imaging: Fast and Cost-effective Morphometry of Musculoskeletal Tissue
08:52

3D Ultrasound Imaging: Fast and Cost-effective Morphometry of Musculoskeletal Tissue

Published on: November 27, 2017

23.2K
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

525
Biomechanical Changes Related to Low Back Pain: An Innovative Tool for Movement Pattern Assessment and Treatment Evaluation in Rehabilitation
06:28

Biomechanical Changes Related to Low Back Pain: An Innovative Tool for Movement Pattern Assessment and Treatment Evaluation in Rehabilitation

Published on: December 13, 2024

452

Area of Science:

  • Orthopaedic Surgery
  • Artificial Intelligence in Medicine
  • Clinical Practice Guidelines

Background:

  • Large language models (LLMs) are increasingly used for medical information retrieval.
  • Assessing the concordance of LLM recommendations with evidence-based clinical practice guidelines (CPGs) is crucial for patient safety.
  • The American Academy of Orthopaedic Surgeons (AAOS) develops CPGs for common orthopaedic conditions.

Purpose of the Study:

  • To evaluate the accuracy of treatment recommendations from leading LLMs against AAOS CPGs for rotator cuff tears and anterior cruciate ligament (ACL) injuries.
  • To compare the concordance rates among different LLMs, including ChatGPT-4, Gemini, Mistral-7B, and Claude-3.

Main Methods:

  • Extracted AAOS CPGs for rotator cuff tears (n=33) and ACL injuries (n=15).
  • Collected treatment recommendations from four LLMs: ChatGPT-4, Gemini, Mistral-7B, and Claude-3.
  • Two blinded physicians assessed LLM recommendations for concordance with AAOS CPGs (concordant, discordant, indeterminate) and analyzed transparency.

Main Results:

  • Overall concordance with AAOS CPGs was 70.3%, with 22.4% indeterminate and 7.3% discordant recommendations.
  • ChatGPT-4 demonstrated the highest concordance (79.2%), while Mistral-7B had the most indeterminate responses (35.4%).
  • Gemini showed the highest rate of discordant recommendations (12.5%), and only 10.4% of LLM responses were transparent with verifiable references.

Conclusions:

  • A significant proportion of LLM recommendations for rotator cuff and ACL injuries do not align with current evidence-based CPGs.
  • While ChatGPT-4 performed best, the observed error rates and lack of transparency indicate LLMs are not yet fully trustworthy clinical support tools.
  • Comparative evaluations of multiple LLMs are necessary to avoid bias and understand individual model strengths and weaknesses.