Related Experiment Video
Updated: Aug 5, 2026
07:48
Eye Tracking During A Complex Aviation Task For Insights Into Information Processing
Published on: April 4, 2025
Dual-Stream Fusion of Eye-Tracking and ECG Signals for Fatigue Detection in Remote Tower Air Traffic Controllers
Dajiang Song1, Weijun Pan1, Hugo Gamboa2
1Key Laboratory of Flight Techniques and Flight Safety, Civil Aviation Flight University of China, Guanghan 618307, China.
Bioengineering (Basel, Switzerland)
|July 28, 2026
Summary
This study introduces MFD-Net, a multimodal fusion framework for detecting fatigue in air traffic controllers using eye-tracking and electrocardiogram (ECG) signals. While promising in controlled tests, further research is needed for reliable real-world fatigue detection.
Area of Science:
- Biomedical Engineering
- Human Factors Engineering
- Cognitive Science
Background:
- Fatigue poses a significant risk to operational safety in air traffic control, particularly in remote towers requiring sustained attention.
- High cognitive workload and prolonged visual monitoring exacerbate controller fatigue.
- Effective fatigue detection is crucial for preventing human errors and ensuring aviation safety.
Purpose of the Study:
- To develop and evaluate MFD-Net, a novel dual-stream multimodal fusion framework for detecting fatigue in remote tower air traffic controllers.
- To assess the performance of MFD-Net using eye-tracking and electrocardiogram (ECG) signals under various validation protocols.
- To investigate the potential of fusion-based approaches for subject-independent fatigue detection and explore calibration strategies.
Main Methods:
- A dual-stream framework (MFD-Net) was designed, processing eye-tracking and ECG data separately before late fusion.
- An expert feature (RMSSD) derived from ECG was incorporated.
- Performance was evaluated using mixed-subject random-window and leave-one-subject-out (LOSO) protocols.
Main Results:
- MFD-Net achieved high accuracy (85.20%) and AUC (0.9337) under a within-distribution protocol.
- Performance significantly decreased under the stricter LOSO protocol (Accuracy: 70.95%, Recall: 22.98%, AUC: 0.6025), indicating limited subject-independent generalization.
- Lightweight calibration improved adaptation but requires cautious interpretation due to random window usage.
Conclusions:
- Fusion of eye-tracking and ECG signals shows promise for fatigue detection in controlled environments.
- Current models exhibit limited subject-independent fatigue detection capabilities, necessitating deployment-oriented calibration.
- Further validation and optimization focusing on recall are essential for practical application in real-world air traffic control settings.
