MRGazer: decoding eye gaze points from functional magnetic resonance imaging in individual space

Area of Science:

• Neuroscience
• Cognitive Science
• Machine Learning

Background:

• Eye-tracking is crucial for understanding cognition.
• Previous deep learning methods for fMRI eye movement analysis required complex co-registration.
• This complexity led to increased processing time and reliance on external templates.

Purpose of the Study:

• To develop an efficient framework for predicting eye gaze from functional magnetic resonance imaging (fMRI) data in individual space.
• To simplify the processing protocol for eye movement analysis from fMRI.
• To enable end-to-end eye gaze regression without fMRI co-registration.

Main Methods:

• Proposed MRGazer framework with an eyeball extraction module and a residual network-based eye gaze prediction module.
• Skipped the fMRI co-registration step, simplifying the protocol.
• Achieved end-to-end eye gaze regression.

Main Results:

• MRGazer achieved superior eye fixation regression performance (Euclidean error, EE = 2.04°) compared to the co-registration-based method (EE = 2.89°).
• The framework delivered results significantly faster (∼0.02 s volume⁻¹) than the prior method (∼0.3 s volume⁻¹).

Conclusions:

• MRGazer is an efficient, simple, and accurate deep learning framework for predicting eye movement from fMRI data.
• The framework can be utilized during fMRI scans in psychological and cognitive research.
• The MRGazer code is publicly available for research use.