An integrated neural network model for eye-tracking during human-computer interaction

Area of Science:

• Human-Computer Interaction
• Aerospace Engineering
• Computer Vision

Background:

• Improving human-computer interaction (HCI) is crucial for intelligent aircraft cockpits.
• Gaze interaction offers a promising method to reduce operator workload and enhance interaction intelligence.
• Effective eye-tracking is fundamental for successful gaze interaction, directly impacting system performance.

Purpose of the Study:

• To develop an accurate eye-tracking method for aircraft cockpit HCI.
• To enable gaze estimation on multiple screens using only facial images.
• To overcome limitations of head movement in operator gaze tracking.

Main Methods:

• A multi-camera system captures facial images, allowing for unrestricted head movement.
• A hybrid neural network combines Transformer for global features and CNN for local features.
• Features from both branches are fused for robust eye-tracking and gaze estimation.

Main Results:

• The proposed method effectively estimates operator gaze position on multiple screens.
• It significantly improves gaze estimation accuracy compared to existing models.
• The system achieves over 80% capture rate for targets of varying sizes.

Conclusions:

• The developed eye-tracking method enhances HCI in aircraft cockpits by enabling natural gaze interaction.
• It provides a solution for accurate gaze estimation despite limited head movement.
• The hybrid network approach demonstrates superior performance and robustness.