Effect of Reporting Mode and Clinical Experience on Radiologists' Gaze and Image Analysis Behavior at Chest...

Background Structured reporting (SR) and artificial intelligence (AI) could potentially transform radiologists' interactions with imaging studies. Purpose To assess the impact of different reporting modes on radiologists' interactions with chest radiographs, with and without AI, regarding image analysis behavior, diagnostic accuracy and efficiency, and user experience. Materials and Methods In this prospective study (July to December 2024), four novice readers (resident radiologists in training and pregraduate medical students) and four non-novice readers (resident radiologists in training) each analyzed 35 bedside chest radiographs using three reporting modes: free-text reporting, SR with itemized and graded findings, and AI-prefilled SR (AI-SR) (ie, SR with AI-prefilled suggestions). A customized viewer displayed radiographs and the reporting interface on a screen-based eye-tracking system. Outcome measures included diagnostic accuracy (compared with the majority vote of six expert radiologists [reference standard], quantified using the Cohen κ coefficient), reporting time per radiograph, eye-tracking metrics, and user experience. Statistical analyses were performed using generalized linear mixed models and Bonferroni post hoc tests, with a significance level of P ≤ .01. Results Chest radiographs from 35 unique patients (mean age, 70.2 years ± 17.1 [SD]; 19 female patients) were included. Diagnostic accuracy was similar in free-text reporting (κ = 0.58) and SR (κ = 0.60; P > .99) but higher in AI-SR (κ = 0.71; both P < .001). Novice readers and non-novice readers both showed improvements with AI-SR compared to free-text reporting (Δκ = 0.17 and 0.09; both P < .001). SR and AI-SR reduced the mean reporting times per radiograph from 88 seconds ± 38 for free-text reporting to 37 seconds ± 18 for SR and 25 seconds ± 9 for AI-SR (both P < .001). Mean saccade counts (quick eye movements between fixations) for radiographs (205.1 ± 134.8 for free-text reporting, 123.1 ± 88.3 for SR, and 96.9 ± 57.8 for AI-SR) and mean total fixation duration for reports (11.4 seconds ± 4.7 for free-text reporting, 4.8 seconds ± 2.6 for SR, and 3.6 seconds ± 0.8 for AI-SR) were lower with SR and AI-SR than with free-text reporting (both P < .001). For novice readers, gaze focus shifted from report to radiograph with SR (percentage of total fixation duration directed towards radiograph, 67.6% for free-text reporting [P = .03 for difference between radiograph and report] vs 74.0% for SR [P = .005]), whereas non-novice readers maintained visual focus on the radiograph regardless of reporting mode (total fixation duration, 8.1 seconds ± 4.4 for free-text reporting, 6.1 seconds ± 3.6 for SR, and 6.5 seconds ± 2.7 for AI-SR; P = .51). Readers were most appreciative of AI-SR (number of readers rating user satisfaction as high: three of eight for free-text reporting, seven of eight for SR, and eight of eight for AI-SR). Conclusion Compared with free-text reporting, SR enhanced efficiency by directing visual attention to the image, while AI-SR improved diagnostic accuracy. © RSNA, 2026 Supplemental material is available for this article.