Automated MRI protocoling in neuroradiology in the era of large language models

Lara Noelle Reiner ¹, Moudather Chelbi ², Leonard Fetscher ²

⁰Department of Neuroradiology, Charité-Universitätsmedizin Berlin, Augustenburger Platz 1, 13353, Berlin, Germany. lara.reiner@charite.de.

La Radiologia Medica +

|

July 11, 2025

View abstract on PubMed

Summary

Large language models (LLMs) show promise in automating MRI protocoling, achieving radiologist-level accuracy for contrast media administration. Retrieval-augmented generation (RAG) significantly boosts LLM performance in predicting MRI sequences and contrast media.

Area Of Science

Artificial Intelligence in Medical Imaging
Radiology Workflow Automation
Natural Language Processing in Healthcare

Background

MRI protocoling is a crucial but time-consuming task in radiology.
Automating this process can improve efficiency and consistency.
Large Language Models (LLMs) offer potential for automating complex medical tasks.

Purpose Of The Study

To investigate the automation of MRI protocoling using LLMs.
To compare the performance of an open-source LLM (LLama 3.1 405B) and a proprietary LLM (GPT-4o).
To evaluate the impact of retrieval-augmented generation (RAG) on LLM accuracy for MRI protocoling.

Main Methods

Retrospective study of 100 MRI studies from 2023.
LLMs were tasked with assigning MRI protocols and contrast media administration.
Performance was evaluated with and without RAG, comparing against a neuroradiologist's gold standard and four radiologists' selections.
Statistical analysis included token-based symmetric accuracy, Wilcoxon signed-rank test, and McNemar test.

Main Results

RAG significantly improved accuracy for both LLama 3.1 and GPT-4o in sequence and contrast media prediction (P < .001).
GPT-4o achieved higher accuracy in MRI sequence prediction (81%) compared to LLama 3.1 (70%), comparable to radiologists (81%).
Both LLMs with RAG demonstrated comparable accuracy to radiologists in predicting contrast media administration (LLama 3.1 RAG: 94%, GPT-4o RAG: 92%).

Conclusions

LLMs demonstrate significant potential as decision-support tools for MRI protocoling.
RAG integration substantially enhances LLM accuracy for institution-specific protocol recommendations.
LLM performance, particularly with RAG, approaches that of experienced radiologists.

Keywords:

Artificial intelligence Automation Clinical Decision-support systems Large language models Magnetic resonance imaging Natural language processing Neuroradiology