Aligning large language models with radiologists by reinforcement learning from AI feedback for chest CT reports

Area of Science:

• Artificial Intelligence in Radiology
• Natural Language Processing for Medical Reports
• Clinical Decision Support Systems

Background:

• Large language models (LLMs) often fail to capture radiologists' nuanced clinical judgment in medical report summarization.
• This limitation can result in radiology reports lacking the quality and professionalism needed for critical diagnostic decisions.

Purpose of the Study:

• To investigate the alignment of LLMs with radiologists using reinforcement learning from AI feedback (RLAIF).
• To enhance the quality and clinical significance of impression summarization in Chest CT reports.

Main Methods:

• A retrospective study analyzed 94,844 chest CT reports.
• LLM-generated impressions were compared and ranked against radiologist impressions by an AI model (Qwen) and a radiologist.
• Further fine-tuning of LLMs was performed using reinforcement learning based on this comparative dataset.

Main Results:

• The AI model (Qwen) achieved a 77.9% agreement rate with the radiologist in ranking report impressions.
• The aligned LLM demonstrated significant performance improvements in Precision (2.56%), Recall (1.77%), and F1 score (1.13%) compared to its unaligned counterpart (P < 0.001).

Conclusions:

• Explicit alignment of LLMs with radiologists using RLAIF effectively improves chest CT report impression generation.
• AI feedback shows potential as a viable alternative to human feedback for aligning LLMs in radiology.
• This study supports the application of AI feedback in clinical CT report summarization and future multi-modal LLM development in radiology.