Search research articles

ABOUT JoVE

Overview Leadership Blog JoVE Help Center

AUTHORS

Publishing Process Editorial Board Scope & Policies Peer Review FAQ Submit

LIBRARIANS

Testimonials Subscriptions Access Resources Library Advisory Board FAQ

RESEARCH

JoVE Journal Methods Collections JoVE Encyclopedia of Experiments Archive

EDUCATION

JoVE Core JoVE Business JoVE Science Education JoVE Lab Manual Faculty Resource Center Faculty Site

Terms & Conditions of Use

Search research articles

Related Experiment Video

Updated: Jul 5, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

Comparing Speed and Accuracy of Artificial Intelligence Large Language Models on the Orthopedic In-Training

Fahad Nadeem¹, Saad Ibrahim¹, Sean Taylor¹

¹Department of Orthopaedic Surgery, University of Alabama at Birmingham, Birmingham.

Southern Medical Journal

|July 3, 2026

Summary

This summary is machine-generated.

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by

Same author

Spinal Specialty Societies: It is Time for Collaboration and Consolidation.

Clinical spine surgery·2026

Same author

Cervical Pedicle Inlet Screws: Clinical Results of a Novel Trajectory for Navigated Screw Placement in the Sub-axial Cervical Spine.

Global spine journal·2026

Same author

Gender inequity in leadership positions in spinal societies worldwide.

European spine journal : official publication of the European Spine Society, the European Spinal Deformity Society, and the European Section of the Cervical Spine Research Society·2026

Same author

Fracture Management and Care in Pregnancy.

JBJS reviews·2026

Same author

Thoracolumbar Instrumentation Options: Biomechanical Implications.

The Journal of the American Academy of Orthopaedic Surgeons·2026

Same author

All-polyethylene, hybrid, and modern metal-backed glenoid components show relevant but not significantly different outcomes in anatomic total shoulder arthroplasty: A systematic review.

Journal of orthopaedics·2026

Same journal

SMA's 6th Annual Physicians-in-Training Leadership Conference Abstract Presentations.

Southern medical journal·2026

Same journal

Potential Impact of Lower Federal Loan Availability on Medical Education in Appalachia.

Southern medical journal·2026

Same journal

Experiential Mentorship Skills Training: The Effect on Real-Life Mentoring.

Southern medical journal·2026

Same journal

Social Pressure: How Early Social Context Shapes Career Interest in Medicine.

Southern medical journal·2026

Same journal

Don't Wait to Talk about Weight: A 2-Hour Interactive Curriculum Improves Medical Student Skills with Weight Management.

Southern medical journal·2026

Same journal

Balancing Relief and Risk: Managing Chronic Low Back Pain in Patients with Opioid Use Disorder.

Southern medical journal·2026

See all related articles

ChatGPT-4 achieved the highest accuracy on orthopedic exams, while ChatGPT-3.5 was the fastest. Other large language models showed lower accuracy and slower response times, indicating potential for AI in orthopedic education.

Area of Science:

Artificial Intelligence in Medicine
Medical Education Technology
Orthopedic Surgery Training

Background:

Large language models (LLMs) show promise in medicine for learning and patient care.
Existing research on LLMs in medicine often focuses on general licensing exams.
Limited data exists on LLM performance in specialized orthopedic assessments.

Purpose of the Study:

To evaluate the accuracy and response speed of leading LLMs on orthopedic-specific examinations.
To compare the performance of ChatGPT-3.5, ChatGPT-4, Microsoft Copilot, and Google Gemini on the Orthopedic In-Training Examination (OITE).

Main Methods:

Utilized questions from the 2020-2022 Orthopedic In-Training Examination (OITE).
Manually input 1582 OITE questions into four LLMs without specific prompts or feedback.

Related Experiment Videos

Last Updated: Jul 5, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

Recorded response accuracy and measured response time from submission to answer generation.

Main Results:

ChatGPT-4 achieved the highest accuracy at 67.09%, significantly outperforming other models (P<0.001).
ChatGPT-3.5 demonstrated the fastest response time at 5.41 seconds.
Both ChatGPT-3.5 and ChatGPT-4 were significantly faster than Gemini and Microsoft Copilot (P<0.001).

Conclusions:

ChatGPT-4 is the most accurate LLM for OITE questions, while ChatGPT-3.5 is the fastest.
Gemini and Microsoft Copilot exhibited lower accuracy and slower response times.
LLMs show potential in orthopedic education, warranting further research into their medical training applications.