Standardizing Heterogeneous MRI Series Description Metadata Using Large Language Models.

Area of Science:

• Medical Imaging and Artificial Intelligence
• Radiology and Machine Learning Applications

Background:

• MRI metadata, specifically free-text series descriptions (SDs), exhibit significant heterogeneity due to variations in input from manufacturers and technologists.
• This variability complicates the accurate identification of MRI sequences, impacting essential tasks like hanging protocol selection and dataset curation.

Purpose of the Study:

• To assess the efficacy of large language models (LLMs) in automatically classifying heterogeneous MRI series descriptions (SDs).
• To evaluate the performance of various LLMs against a neuroradiologist's ground truth classification of MRI SDs.

Main Methods:

• Analysis of non-contrast brain MRI examinations conducted between 2016 and 2022.
• Manual classification of unique series descriptions (SDs) into predefined categories (T1, T2, T2/FLAIR, SWI, DWI, ADC, Other) by a practicing neuroradiologist.
• Performance evaluation of multiple LLMs (GPT 3.5 Turbo, GPT-4, GPT-4o, Llama 3 8b, Llama 3 70b) using area under the curve (AUC) as the primary metric, with GPT-4o also tasked with generating regular expression templates.

Main Results:

• A high degree of variability was observed, with 52.1% of 1395 unique SDs appearing only once.
• GPT-4o achieved the highest performance, with an average AUC of 0.983 ± 0.020 for all series when provided with detailed prompts.
• GPT models significantly outperformed Llama models; regular expression generation by GPT-4o showed inconsistent performance (AUC of 0.774 ± 0.161).

Conclusions:

• Large language models (LLMs) demonstrate significant effectiveness in interpreting and standardizing heterogeneous MRI series descriptions (SDs).
• The study highlights the potential of LLMs to enhance the accuracy and efficiency of MRI data management and analysis.