Automated MRI pituitary structured reporting from free-text using a fine-tuned Llama model: a feasibility study

Structured reports in radiology have demonstrated substantial advantages over unstructured ones. However, the transition from unstructured to structured reporting can face challenges, as experienced radiologists worry about the potential loss of valuable information. In this study, we fine-tuned the Llama 2 model capable of generating structured pituitary MRI reports from unstructured reports.

We used a training set comprising 104 pituitary MRI reports to fine-tune Llama 2 and 26 reports as a test set to evaluate the system. The dataset was annotated manually by three expert radiologists. For this annotation, the radiologists used the unstructured report and structured it into eight anatomical landmarks: adenohypophysis, pituitary stalk, optic chiasm, suprasellar cistern, neurohypophysis, cavernous sinuses, sphenoid sinuses and other findings.

Llama2 achieves a value greater than 0.79 on the ROUGE-L metric in four anatomical landmarks from free-text pituitary MRI reports. The other anatomical landmarks exceed 0.61 of ROUGE-L except for the other findings section.

Our study suggests good performance in structuring anatomical landmarks on pituitary MRI reports using the fine-tune Llama 2 model.