Modeling radiologists' cognitive processes using a digital gaze twin to enhance radiology training

Area of Science:

• Artificial Intelligence
• Medical Imaging
• Cognitive Science

Background:

• Predicting human gaze behavior is crucial for advancing interactive systems and improving diagnostic accuracy in medical imaging.
• Understanding radiologists' cognitive processes during image interpretation is key to enhancing diagnostic tools.

Purpose of the Study:

• To introduce MedGaze, a novel system inspired by the "Digital Gaze Twin" concept.
• To model radiologists' cognitive processes and predict scanpaths in chest X-ray (CXR) images.
• To evaluate MedGaze's performance against state-of-the-art methods and its ability to assess clinical workload.

Main Methods:

• MedGaze utilizes a two-stage training approach: Vision to Radiology Report Learning (VR2) and Vision-Language Cognition Learning (VLC).
• The system combines visual features with radiology reports, leveraging large datasets like MIMIC to replicate radiologists' visual search patterns.
• Performance was evaluated on EGD-CXR and REFLACX datasets using IoU, Correlation Coefficient (CC), and Multimatch scores. Clinical workload was assessed via fixation duration and Spearman rank correlation.

Main Results:

• MedGaze significantly outperformed state-of-the-art methods on EGD-CXR and REFLACX datasets, achieving higher IoU, CC, and Multimatch scores.
• Fixation duration analysis showed a significant correlation with clinical workload, indicating MedGaze's utility in workload assessment.
• Human evaluation demonstrated that predicted scanpaths closely resembled expert patterns and covered key diagnostic regions.

Conclusions:

• MedGaze effectively predicts radiologist gaze behavior on CXR images, emulating expert visual search patterns.
• The system offers valuable insights into radiologist decision-making, enhancing training and diagnostic accuracy.
• MedGaze has the potential to improve clinical outcomes by minimizing redundancy and optimizing diagnostic workflows.