Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Concept Videos

Stereotype Content Model02:16

Stereotype Content Model

The Stereotype Content Model (SCM) was first proposed by Susan Fiske and her colleagues (Fiske, Cuddy, Glick & Xu, 2002; see also Fiske, 2012 and Fiske, 2017). The SCM specifies that when someone encounters a new group, they will stereotype them based on two metrics: warmth—or that group’s perceived intent, and how likely they are to provide help or inflict harm—and competence—or their ability to carry out that objective. Depending on the warmth-competence categorization, a person will feel...

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Large Language Models Should Be the New Gold Standard for Orthopaedic Surgical Registry Curation, but Likely Will Not Be without Clearer Direction: Commentary on an article by Linjun Yang, PhD, et al.: "Comparison of Large Language Models with Rules-Based Natural Language Processing Algorithms for Extracting Data from Operative Notes".

The Journal of bone and joint surgery. American volume·2026
Same author

Age-Stratified Imaging Selection in Langerhans Cell Histiocytosis: Towards a Clinical Decision Framework.

Journal of clinical medicine·2026
Same author

Machine Learning Better Predicts Mortality in Patients Undergoing Revision Total Hip Arthroplasty for Periprosthetic Joint Infection.

Journal of orthopaedic research : official publication of the Orthopaedic Research Society·2026
Same author

Personalization of Physical Activity Counseling in Knee Osteoarthritis Requires Precise Data, Not Precise Statistics: Commentary on an article by Qianying Cai, PhD, et al.: "Physical Activity and 4-Year Radiographic Medial Joint Space Loss in Knee Osteoarthritis. A Joint Model Analysis".

The Journal of bone and joint surgery. American volume·2026
Same author

Tumor Size Modifies the Survival Benefit of Chemotherapy in Localized Soft Tissue Sarcomas: A Propensity-Matched Cohort Study.

Journal of clinical medicine·2026
Same author

Low Survival Rates for Pediatric Patients with Tumor Thrombus in Sarcoma.

Journal of clinical medicine·2026

Related Experiment Video

Updated: Jun 17, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

Custom and Off-the-Shelf Large Language Models Routinely Misinterpret Implant Technique Guides: Too Soon to

Joshua J Woo1, Andrew J Yang1, Yash S Saboo2

  • 1The Warren Alpert Medical School of Brown University, Providence, Rhode Island.

JB & JS Open Access
|June 16, 2026
PubMed
Summary
This summary is machine-generated.

Large language models (LLMs) show poor accuracy in extracting surgical information from orthopaedic device instructions. Advanced AI is not yet ready to replace medical device representatives in surgical settings.

More Related Videos

Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
05:47

Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems

Published on: June 13, 2025

Related Experiment Videos

Last Updated: Jun 17, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems
05:47

Evidence-based Knowledge Synthesis and Hypothesis Validation: Navigating Biomedical Knowledge Bases via Explainable AI and Agentic Systems

Published on: June 13, 2025

Area of Science:

  • Orthopaedic surgery
  • Artificial intelligence
  • Medical device technology

Background:

  • Large language models (LLMs) are increasingly used for clinical information retrieval.
  • Their efficacy with highly specialized documents like orthopaedic technique guides and instructions for use (IFU) is not well understood.
  • Financial pressures in the orthopaedic device industry drive interest in generative AI for perioperative support.

Purpose of the Study:

  • To assess the clinical suitability of custom LLM applications for replacing manufacturer-specific IFUs in surgical planning and execution.
  • To determine if LLM-based systems can reliably extract complex procedural information from orthopaedic implant protocols.

Main Methods:

  • Evaluated 5 LLM information retrieval solutions (4 custom pipelines, ChatGPT) using 3 distal femoral replacement IFUs.
  • Two orthopaedic surgeons developed 28 questions (literal, enumerative, reasoned).
  • Answers were scored against expert ground truth using a three-tier rubric (incorrect, partially correct, fully correct).

Main Results:

  • All evaluated LLM systems achieved overall accuracy below 50%.
  • A custom multimodal pipeline scored highest (44.6%), outperforming ChatGPT (29.2%).
  • Performance varied by query type, with literal questions answered most accurately (up to 53.0%) and reasoned questions least accurately (as low as 15.3%).

Conclusions:

  • Current LLM retrieval systems are unreliable for complex procedural information in orthopaedic implant protocols.
  • Generative AI is not yet capable of replacing medical device representatives due to performance limitations in surgical workflows.
  • Improvements in LLM image processing and domain-specific training are needed for potential future substitution.