Retrieval-augmented generation-enhanced large language models for comprehensive CAD-RADS 2.0 categorization from...

Area of Science:

• Artificial Intelligence in Medical Imaging
• Radiology Informatics
• Natural Language Processing in Healthcare

Background:

• Structured reporting in coronary computed tomography angiography (CCTA) is crucial for accurate diagnosis and management.
• Large Language Models (LLMs) offer potential for automating data extraction from clinical reports.
• Evaluating LLM performance in extracting specific components and recommendations from CCTA reports is necessary.

Purpose of the Study:

• To assess the performance of standard and retrieval-augmented generation (RAG)-based LLMs in extracting components and management recommendations from CCTA reports.
• To compare the accuracy of different LLMs, including ChatGPT-5, NotebookLM, and a RAG-adapted ChatGPT-5, against expert radiologists' assessments.
• To evaluate the utility of LLMs in adhering to the Coronary Artery Disease Reporting and Data System (CAD-RADS 2.0) guidelines.

Main Methods:

• Analysis of 320 structured CCTA reports using three LLMs: standard ChatGPT-5, NotebookLM (RAG-based), and ChatGPT-5-RAG.
• Extraction of CAD-RADS category, plaque burden, high-risk plaque (HRP), modifiers, full score, and management recommendations.
• Comparison of LLM outputs against a reference standard established by two expert cardiovascular radiologists.

Main Results:

• ChatGPT-5-RAG demonstrated superior accuracy across all evaluated CAD-RADS 2.0 components, including classification, plaque burden, HRP detection, and modifiers.
• Standard ChatGPT-5 exhibited the weakest performance among the tested models.
• While agreement on management recommendations was low, ChatGPT-5-RAG and NotebookLM achieved near-perfect qualitative ratings.

Conclusions:

• RAG-enhanced LLMs significantly improve the accuracy and reliability of extracting CAD-RADS 2.0 components and generating management recommendations.
• RAG-based LLMs represent promising tools for automating and standardizing CCTA reporting within clinical radiology workflows.
• The explainability and innovation offered by RAG-based LLMs can enhance clinical decision-making in cardiovascular imaging.