Using large language model to aid in teaching medical imaging report writing

This study aims to compare several free large language models (LLMs), identify which provides the most effective feedback, and investigate whether LLM-generated feedback can improve the accuracy and standardization of imaging reports produced by students.

A randomly selected class (test group, N= 30) was asked to write an imaging report based on each typical teaching case before and after receiving feedback generated by LLM. Another randomly selected class (control group, N= 30) was asked to write an imaging report of the same case without receiving the LLM-generated feedback. The quality of the feedback generated by the 4 main free LLMs was evaluated. The residency training examination marking scale was used to evaluate the quality of the reports. A questionnaire was used to investigate whether the students were satisfied with the feedback given by LLM.

The feedback generated by ChatGPT 3.5, ERNIE Bot v3.5, and Tongyi v2.5 all demonstrated better structure and logic than that of Claude 3 OPUS (Mann-Whitney U Test, p < 0.05), but all exhibited some degree of hallucination. The scores of the reports in the test group were increased after receiving the feedback, and were higher than the control group (t-test, p < 0.05).

The feedback given by LLMs can help the students critically evaluate their reports and improve their reporting skills, but should be supervised by teachers.