Remote Tower Air Traffic Controller Multimodal Fatigue Detection.

Area of Science:

• Human Factors and Aviation Safety
• Biomedical Engineering and Signal Processing
• Artificial Intelligence in Critical Systems

Background:

• Remote tower (rTWR) operations present human-factor risks, particularly cognitive fatigue from visual surveillance.
• Missed detections in air traffic control are critical, necessitating effective fatigue mitigation strategies.
• Existing methods may not adequately address the nuances of fatigue in safety-critical, screen-based environments.

Purpose of the Study:

• To develop and validate a non-intrusive, multimodal fatigue detection framework for air traffic controllers.
• To fuse ocular and cardiac signals for robust fatigue monitoring in remote tower operations.
• To address challenges of data imbalance and scarcity in physiological fatigue detection.

Main Methods:

• Collected eye-tracking and electrocardiogram (ECG) data from 36 air traffic controllers in a high-fidelity simulation.
• Extracted a 12-dimensional feature vector including gaze entropy and heart rate variability (HRV).
• Developed a cost-sensitive XGBoost classifier using SMOTE oversampling and a dynamically weighted loss function to handle data imbalance.

Main Results:

• The proposed framework demonstrated good performance in mixed-subject evaluations, improving sensitivity to fatigue events.
• A performance drop was noted in leave-one-subject-out (LOSO) evaluations, indicating variability in individual responses.
• Personalized calibration was found to partially mitigate the performance decrease in LOSO evaluations.

Conclusions:

• The multimodal fatigue detection framework shows promise for real-time monitoring in remote tower operations.
• Fusion of ocular and cardiac signals offers a viable approach to detecting cognitive fatigue.
• Personalized calibration is crucial for optimizing the framework's effectiveness across different individuals.