Improving Self-Supervised Medical Image Pre-Training by Early Alignment With Human Eye Gaze Information

Area of Science:

• Artificial Intelligence
• Machine Learning
• Computer Vision

Background:

• Aligning human knowledge with machine learning models is essential for efficient and interpretable AI.
• Current self-supervised pre-training methods lack early human knowledge integration, relying on post-hoc alignment.
• This leads to suboptimal efficiency and interpretability in AI systems.

Purpose of the Study:

• To introduce Gaze Pre-Training (GzPT), a novel approach for early alignment of self-supervised models with human knowledge.
• To enhance the learning efficiency and performance of self-supervised models by incorporating human eye gaze data.
• To investigate the potential of human eye gaze as a passive knowledge source for bridging the human-AI gap.

Main Methods:

• GzPT utilizes contrastive learning to align images based on similar human eye gaze patterns during pre-training.
• The method integrates human eye gaze information early in the self-supervised pre-training process.
• Effectiveness was validated on three diverse medical image datasets.

Main Results:

• GzPT consistently outperformed baseline methods in learning efficiency and performance.
• The approach enabled self-supervised models to learn more meaningful and interpretable representations.
• Early alignment with human eye gaze significantly improved model alignment with human attention.

Conclusions:

• Gaze Pre-Training (GzPT) offers an effective strategy for enhancing self-supervised learning through early integration of human eye gaze.
• Incorporating human eye gaze data provides a valuable form of passive knowledge, improving AI model interpretability and efficiency.
• This method demonstrates a promising direction for developing more human-aligned and understandable AI systems.