Jove
Visualize
Contact Us
JoVE
x logofacebook logolinkedin logoyoutube logo
ABOUT JoVE
OverviewLeadershipBlogJoVE Help Center
AUTHORS
Publishing ProcessEditorial BoardScope & PoliciesPeer ReviewFAQSubmit
LIBRARIANS
TestimonialsSubscriptionsAccessResourcesLibrary Advisory BoardFAQ
RESEARCH
JoVE JournalMethods CollectionsJoVE Encyclopedia of ExperimentsArchive
EDUCATION
JoVE CoreJoVE BusinessJoVE Science EducationJoVE Lab ManualFaculty Resource CenterFaculty Site
Terms & Conditions of Use
Privacy Policy
Policies

Related Experiment Video

Updated: Jun 16, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

Item-Level Evaluation of Multimodal Large Language Models in Neuroradiology: Generational Performance and Execution

Jose Federico Ojeda-Esparza1, Yeong-Chul Yun1, Clara Meinzer1

  • 1From the Division of Neuroradiology (J.F.O.-E., C.S., D.B., A.F., K.-O.L., F.T.K.), Radiology (M.P., M.S., N.V., N.R.), Geneva University Hospitals, Geneva, Switzerland; Faculty of Medicine (J.F.O.-E., C.S., D.B., A.F., M.P., M.S., K.-O.L.), University of Geneva, Geneva, Switzerland; Division of Radiology (C.M.), German Cancer Research Center, Neuroradiology (Y.-C.Y.), Heidelberg University Hospital, Heidelberg, Germany.

AJNR. American Journal of Neuroradiology
|June 13, 2026
PubMed
Summary

Related Concept Videos

You might also read

Related Articles

Articles linked to this work by shared authors, journal, and citation graph.

Sort by
Same author

Seizure relapse in new onset epilepsy: It is not always drug resistance.

Epilepsia·2026
Same author

IDH status shapes glioma oncotopy: voxel-wise mapping of 644 adult diffuse gliomas.

Neuroradiology·2026
Same author

Incorporating functional soft tissue deformations in AI model training for spatially accurate prostate cancer detection.

Magnetic resonance imaging·2026
Same author

Diabetic Peripheral Neuropathy: Current Epidemiology, Diagnostic Advances, Biomarkers, and Management Strategies.

Journal of diabetes research·2026
Same author

Beyond the Leak: Advanced MRI Assessment of the Blood-Brain Barrier in Neurodegeneration.

Neuro-degenerative diseases·2026
Same author

Voxel-accurate MRI-microscopy Correlation Enables AI-powered Prediction of Brain Disease States.

Theranostics·2026

Large language models show progress in neuroradiology tasks but do not match expert performance. Item-level evaluations reveal a persistent gap, highlighting the need for consistency assessment beyond aggregate accuracy.

Area of Science:

  • Artificial Intelligence in Medicine
  • Medical Imaging Analysis
  • Neuroradiology

Background:

  • Multimodal large language models (LLMs) show promise in medical tasks, including radiology.
  • Uncertainty exists regarding LLM convergence with expert performance in specialized domains like neuroradiology.
  • Item-level, human-referenced evaluations are crucial for assessing true expert alignment.

Purpose of the Study:

  • To compare the performance of advanced vision-capable large language models against expert neuroradiologists and residents.
  • To evaluate whether current LLMs achieve expert-level accuracy in neuroradiology using item-level analysis.
  • To assess the consistency and reliability of LLM performance in neuroradiology tasks.

Main Methods:

  • Comparison of four vision-capable LLMs (GPT-4, GPT-5, Gemini 1.5, Gemini 2.5) with expert neuroradiologists and residents.

More Related Videos

Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques
08:05

Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques

Published on: June 30, 2020

Related Experiment Videos

Last Updated: Jun 16, 2026

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
03:14

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Published on: December 6, 2024

Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques
08:05

Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques

Published on: June 30, 2020

  • Utilized 106 image-based neuroradiology multiple-choice questions from Radiopaedia.
  • Employed item-level analysis with paired comparisons and non-parametric bootstrap confidence intervals for accuracy assessment.

    • Main Results:

    • Expert neuroradiologists achieved the highest accuracy (0.915).
    • Second-generation LLMs showed improved accuracy, nearing or exceeding resident performance in some cases.
    • A significant performance gap persisted between advanced LLMs and expert neuroradiologists, with models aligning more with aggregate learner performance.

    Conclusions:

    • Generational gains in LLMs for neuroradiology are evident but do not represent convergence with expert performance.
    • Item-level evaluation is critical for distinguishing genuine performance alignment from benchmark improvements.
    • Aggregate accuracy stability does not guarantee reliability in individual decision-making, necessitating assessment of both performance and consistency.