Nonlinear multi-head cross-attention network and programmable gradient information for gaze estimation.

Area of Science:

• Computer Vision
• Machine Learning
• Human-Computer Interaction

Background:

• Gaze estimation is crucial for human behavior analysis and assistance.
• Current methods using Convolutional Neural Networks (CNNs) and attention Transformers have limitations in feature extraction and utilization.
• CNNs capture limited local context, while attention mechanisms underutilize multiscale features.

Purpose of the Study:

• To propose a novel nonlinear multi-head cross-attention network with programmable gradient information (MCA-PGI) for enhanced gaze estimation.
• To overcome the limitations of existing CNN and Transformer-based gaze estimation methods.
• To improve the accuracy and robustness of gaze estimation systems.

Main Methods:

• Developed a novel nonlinear multi-head cross-attention network with programmable gradient information (MCA-PGI).
• Integrated CNNs and Transformer advantages, utilizing an auxiliary branch for gradient information.
• Employed nonlinear multi-head cross-attention to fuse global visual and multiscale hybrid features.

Main Results:

• MCA-PGI demonstrated strong competitiveness against state-of-the-art methods.
• Achieved significant performance improvements: 2.5% on MPIIFaceGaze and 10.2% on Eyediap datasets.
• The proposed method effectively retains original information and fuses multiscale features.

Conclusions:

• The MCA-PGI network offers a superior approach to gaze estimation by effectively combining local and global feature extraction.
• The method shows significant potential for applications requiring accurate human behavior analysis.
• Open-source implementation is available for further research and development.