A multi-query, multimodal, receiver-augmented solution to extract contemporary cardiology guideline information using...

Area of Science:

• Cardiology
• Artificial Intelligence
• Clinical Decision Support

Background:

• Clinicians require timely access to evidence-based, point-of-care guidelines for optimal patient treatment.
• Existing large language models (LLMs) may lack the specificity and transparency needed for clinical practice.
• Integrating clinical practice recommendations into LLMs is crucial for reliable healthcare applications.

Purpose of the Study:

• To evaluate a novel LLM system designed to deliver transparent, guideline-based cardiology recommendations.
• To assess the LLM's utility in supporting clinicians with individual patient treatment decisions.
• To compare the performance of the developed LLM against established models like GPT-3.5 and GPT-4.

Main Methods:

• Developed a retrieval-augmented generation chain using cloud-based and local LLMs with open-source tools.
• Ensured the system closely mirrors European cardiology guidelines.
• Compared the system's performance against GPT-3.5 and GPT-4 using a 306-question multiple-choice cardiology exam.

Main Results:

• The developed LLM system achieved an overall accuracy of 73.53% on the cardiology exam.
• GPT-3.5 and GPT-4 achieved accuracies of 44.03% and 62.26%, respectively.
• The system demonstrated superior performance across multiple cardiology subspecialties, including coronary artery disease, arrhythmia, and valvular heart disease.

Conclusions:

• The tailored LLM system significantly outperforms current public chat models in cardiology question-answering.
• LLMs can be effectively adapted to provide documented, accountable, and traceable guideline recommendations for clinical use.
• This approach enhances transparency and trustworthiness in AI-driven clinical decision support.