Less Is More: Radiologists Prefer Brief Narratives in LLM-Generated Clinical History Summaries

Clinical histories are essential for accurate radiologic interpretation but are often lengthy and time-consuming to review amid increasing radiologist workloads. Large language models (LLMs) offer a potential solution by generating concise, clinically focused summaries from existing documentation; however, the optimal presentation format for radiologists remains unclear. This retrospective reader study evaluated the readability, efficiency, and radiologist preference for a two-stage LLM-based summarization pipeline designed to identify the optimal summary format for radiologic use. Ninety imaging studies across multiple modalities and care settings were processed to generate structured timeline summaries (stage 1) and brief narrative summaries (stage 2). Text length was reduced by approximately 86.8% and 95.2% for stage 1 and stage 2, respectively. Four radiologists independently rated representative subsets of cases for clarity, relevance, usefulness, and comprehensiveness and indicated their order of preference. Stage 2 summaries were preferred in 89% of ratings and were rated significantly higher than stage 1 for perceived clarity, relevance, and usefulness (p < 0.0001), with equivalent comprehensiveness. Longer source documents and longer stage 1 summaries were associated with lower ratings for the detailed timeline format, whereas no such associations were observed for the brief narrative summaries. Manual review identified 25 total hallucinations across 90 cases, of which one was considered capable of altering image interpretation in a case with borderline radiographic features. These findings demonstrate that summary format, and particularly brevity, is critical for optimizing readability and radiologist preference. Automated summarization holds promise to reduce cognitive load and improve information accessibility in radiology practice.