Ultrasound Image Representation Learning by Modeling Sonographer Visual Attention.

Area of Science:

• Medical Imaging
• Computer Vision
• Artificial Intelligence

Background:

• Current image representation learning relies heavily on manual class labels, which are limited.
• Modeling human visual attention offers a potential avenue for unsupervised representation learning.

Purpose of the Study:

• To demonstrate transferable image representations learned from visual attention without manual annotations.
• To evaluate the efficacy of attention-based models for ultrasound standard plane detection.

Main Methods:

• Trained convolutional neural networks (CNNs) to predict sonographer gaze on ultrasound video frames.
• Utilized visual saliency prediction and gaze-point regression for attention modeling.
• Evaluated transferability via fine-tuning CNNs and analyzing feature activations.

Main Results:

• Fine-tuning a saliency predictor improved F1-scores by 9.6% overall and 15.3% for cardiac planes compared to random initialization.
• Attention models derived strong representations, nearing fully-supervised baseline performance.

Conclusions:

• Learning image representations from visual attention is a viable unsupervised method for medical imaging.
• Attention-based models show significant promise for improving diagnostic accuracy in ultrasound screenings.