Eye Tracking for Deep Learning Segmentation Using Convolutional Neural Networks

Area of Science:

• Medical Imaging
• Deep Learning
• Computer Vision

Background:

• Convolutional Neural Networks (CNNs) show high accuracy in medical image semantic segmentation.
• A major limitation is the need for large-scale, precisely annotated imaging datasets for training.
• Eye tracking (ET) technology is explored as a novel method to address this data annotation bottleneck.

Purpose of the Study:

• To evaluate if segmentation masks generated by eye tracking (ET) are similar to hand annotations (HA).
• To determine if a CNN trained on ET masks is equivalent to one trained on HA masks.
• To assess the feasibility of ET for efficient medical image annotation.

Main Methods:

• Two steps were used: 1) Analysis of 19 public radiologic images for ET and HA mask generation. 2) Generation of ET and HA masks for 356 meningioma images.
• A U-net based CNN was trained on 306 image-mask pairs (ET and HA), with 50 images reserved for testing.
• Dice Similarity Coefficient (DSC) and Area Under the Curve (AUC) were used to compare mask and model performance.

Main Results:

• ET and HA masks showed high similarity, with average DSC of 0.86 for non-neurological images and 0.85 for meningioma images.
• CNNs trained with ET and HA masks achieved comparable performance on the test set (AUC 0.88 vs. 0.87).
• Trimmed DSCs between ET and HA predictions were statistically equivalent (p=0.015), indicating similar segmentation accuracy.

Conclusions:

• Eye tracking (ET) technology can generate segmentation masks suitable for deep learning semantic segmentation in medical imaging.
• ET offers a viable alternative to manual annotation, potentially accelerating the training of CNNs.
• Future research will focus on integrating ET into clinical workflows for faster and more natural mask generation.