Automated MRI pituitary structured reporting from free-text using a fine-tuned Llama model: a feasibility study

Area of Science:

• Radiology and Artificial Intelligence
• Natural Language Processing in Medical Imaging

Background:

• Structured radiology reports offer significant advantages over unstructured ones.
• Transitioning to structured reporting can be challenging, with concerns about information loss.

Purpose of the Study:

• To fine-tune the Llama 2 model for generating structured pituitary MRI reports from unstructured text.
• To evaluate the model's performance in extracting anatomical landmarks.

Main Methods:

• A dataset of 104 pituitary MRI reports was used to fine-tune Llama 2.
• An additional 26 reports served as a test set for evaluation.
• Expert radiologists manually annotated eight anatomical landmarks.

Main Results:

• Llama 2 achieved a ROUGE-L score greater than 0.79 for four anatomical landmarks.
• Performance exceeded 0.61 ROUGE-L for most landmarks, excluding 'other findings'.

Conclusions:

• The fine-tuned Llama 2 model shows promising performance in structuring anatomical landmarks from pituitary MRI reports.
• This approach can aid in the transition to structured reporting.