Related Experiment Video
Updated: Sep 11, 2025
Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness
Published on: December 6, 2024
Performance Evaluation of Large Language Models With Retrieval-Augmented Generation in Cardiology Specialist
Hiromasa Hayama1,2, Tu Hao Tran3, Jin Kirigaya1,2
1School of Clinical Medicine, University of New South Wales Sydney, NSW Australia.
Background:
Large language models (LLMs) have shown potential in medical education, but their application to cardiology specialist examinations remains underexplored. We compared the performances of a retrieval-augmented generation LLM (RAG-LLM) 'CardioCanon' against general-purpose LLMs.
Methods And Results:
A total of 96 publicly available text-based open-source multiple-choice questions from the Japanese Cardiology Specialist Examination (1997-2022) were used. CardioCanon showed similar option-level accuracy to ChatGPT-4o and Gemini 2.0 Flash (81.0%, 76.0%, and 77.2%, respectively), but higher case-based accuracy than ChatGPT (57.3% vs. 29.2%, P<0.001).
Conclusions:
RAG techniques can enhance AI-assisted examination performance by improving case-level reasoning and decision-making.
More Related Videos
Related Concept Videos
02:07Improving Translational Accuracy
01:15Myasthenia Gravis: Diagnostic Tests
The edrophonium test is a diagnostic tool for myasthenia gravis. It involves...