Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Video

Updated: Jun 29, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

Comparative Evaluation of Large Language Models for Surgical Case Creation.

Connie Y Gan1, Megan Chan2, Nitasha Sharma3

  • 1Department of Surgery, Stanford University, Stanford, California; Department of Surgery, Oregon Health and Science University, Portland, Oregon.

Journal of Surgical Education
|June 27, 2026
PubMed
Summary

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same authorSame journal

Advising Through the Noise: Medical Student Advisor Perspectives on How Preference Signals Have Influenced Surgical Residency Recruitment.

Journal of surgical education·2026
Same author

Inflammaging and osteoarthritis - therapeutic opportunities.

Frontiers in immunology·2026
Same authorSame journal

Mixed Signals: General Surgery Residency Applicant Attitudes Toward Preference Signaling.

Journal of surgical education·2026
Same authorSame journal

Thanks for the Signal-Now What? Program Director Perspectives on How Preference Signals Are Shaping General Surgery Residency Recruitment.

Journal of surgical education·2026
Same author

MiR-425-5p modulation of CREB1 affects inflammatory response and motor recovery after spinal cord injury.

Biochemical and biophysical research communications·2026
Same author

Knowledge, Attitudes, and Practices of Adult Vaccination Among Medical Students in Western India.

Cureus·2026

Claude-3.5-sonnet excelled in generating trauma surgery case scenarios for surgical education. This study highlights the importance of selecting the right large language model (LLM) for AI-driven medical training content.

Area of Science:

  • Artificial Intelligence in Medical Education
  • Surgical Simulation Development
  • Natural Language Processing in Healthcare

Background:

  • Virtual simulation platforms like ENTRUST are crucial for surgical trainee education.
  • Assessing clinical decision-making requires realistic and high-quality case scenarios.
  • Large language models (LLMs) offer potential for automated content generation in medical education.

Purpose of the Study:

  • To compare the performance of five leading LLMs in generating trauma surgery case scenarios.
  • To evaluate the quality and usability of AI-generated content for surgical training.
  • To introduce a modified HumanELY framework for assessing AI-generated educational materials.

Main Methods:

  • A comparative experimental analysis was conducted using a standardized prompt across five LLMs: ChatGPT-4o, Claude-3.5-sonnet, Gemini-2.0-flash-001, Llama-3.2, and DeepSeek-r1.
Keywords:
artificial intelligenceclinical decision-makinggeneral surgerylarge language modelssimulation trainingsurgical education

Related Experiment Videos

Last Updated: Jun 29, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

  • Outputs were evaluated by board-certified trauma surgeons using modified HumanELY metrics (relevance, coverage, coherence, lack of harm), usability, and readability.
  • Reference materials included American Board of Surgery EPA definitions and ENTRUST platform guidelines.

    • Main Results:

    • Claude-3.5-sonnet achieved the highest mean total modified HumanELY score (52.5/70), significantly outperforming other models (p=0.04).
    • Gemini-2.0-flash-001, ChatGPT-4o, and DeepSeek-r1 showed similar performance (46.7-45.8/70), while Llama-3.2 scored lowest (39.8/70).
    • Claude-3.5-sonnet required the least post-generation editing (22.5%), indicating superior output quality and usability.

    Conclusions:

    • Performance of LLMs in generating surgical case content varies significantly.
    • Claude-3.5-sonnet demonstrated superior capability in producing high-quality trauma surgery cases for simulation.
    • Careful LLM selection and the use of structured evaluation frameworks are essential for effective AI integration in surgical education.