Spatio-temporal visual attention modelling of standard biometry plane-finding navigation.

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Biomedical Engineering

Background:

• Accurate fetal biometry is crucial for assessing fetal growth and well-being.
• Understanding sonographer visual attention can optimize ultrasound training and improve efficiency.
• Current methods for analyzing sonographer gaze patterns in ultrasound are limited.

Purpose of the Study:

• To introduce Temporal SonoEyeNet (TSEN), a multi-task neural network designed to model the visual navigation of sonographers during ultrasound examinations.
• To generate visual attention maps around standard fetal biometry planes (abdomen, head, femur).
• To validate the clinical relevance of predicted visual attention maps in guiding biometry plane detection.

Main Methods:

• TSEN incorporates a feature extractor, a temporal attention module (TAM), and a video classification module (VCM).
• A soft dynamic time warping (sDTW) loss function was employed for enhanced visual attention modeling.
• Models were trained on 280 ultrasound video clips with corresponding sonographer gaze tracking data.

Main Results:

• The best performing TSEN variant, utilizing bi-directional convolutional long-short term memory (biCLSTM), outperformed a previous spatial model on saliency metrics.
• Spatio-temporal TSEN models significantly improved standard biometry plane detection compared to a spatial-only baseline.
• The leading TSEN model achieved F1 scores of 83.7% (abdomen), 89.9% (head), and 81.1% (femur) for biometry plane detection.

Conclusions:

• TSEN effectively models sonographer visual attention during fetal ultrasound biometry.
• The AI-driven visual attention maps can guide and improve the accuracy of standard biometry plane detection.
• This work offers a novel approach for analyzing and potentially automating aspects of ultrasound examinations.