First Trimester video Saliency Prediction using CLSTMU-NET with Stochastic Augmentation.

Area of Science:

• Medical Imaging
• Artificial Intelligence
• Computer Vision

Background:

• First-trimester fetal ultrasound scanning relies on sonographer expertise to interpret complex visual information.
• Predicting sonographer gaze can optimize training and improve diagnostic accuracy in ultrasound procedures.

Purpose of the Study:

• To develop and evaluate a multi-modal video analysis algorithm for predicting sonographer gaze direction in first-trimester ultrasound videos.
• To assess the performance of a spatio-temporal convolutional LSTM U-Net (cLSTMU-Net) against spatial-only models.

Main Methods:

• Utilized gaze tracking data from expert sonographers performing routine first-trimester fetal ultrasounds.
• Developed a cLSTMU-Net architecture combining a U-Net encoder-decoder with a convolutional LSTM (cLSTM) for spatio-temporal analysis.
• Employed a Random Augmentation (RA) strategy for model training and to mitigate overfitting.
• Compared cLSTMU-Net performance against baseline spatial-only architectures using saliency metrics (KLD, SIM, NSS, CC).

Main Results:

• The proposed cLSTMU-Net demonstrated superior performance across all evaluated saliency metrics compared to spatial-only approaches.
• Specifically, cLSTMU-Net achieved better scores in KLD (2.08 vs 2.16), SIM (0.28 vs 0.27), NSS (4.53 vs 4.34), and CC (0.42 vs 0.39) using a 6-frame video clip.
• The model was trained on a dataset of 115 ultrasound videos (45,666 frames).

Conclusions:

• The cLSTMU-Net effectively predicts sonographer gaze in ultrasound videos, outperforming traditional methods.
• This AI-driven approach holds potential for enhancing ultrasound training and real-time guidance systems.
• Spatio-temporal analysis is crucial for accurate gaze prediction in dynamic medical imaging contexts.