FatigueNet: A hybrid graph neural network and transformer framework for real-time multimodal fatigue detection

Area of Science:

• Biomedical Engineering
• Artificial Intelligence
• Signal Processing

Background:

• Fatigue presents complex challenges impacting cognitive, physical, and emotional well-being.
• Current fatigue classification models struggle with biosignal diversity and interdependence.
• There is a need for advanced frameworks to accurately detect and monitor fatigue levels.

Purpose of the Study:

• To introduce FatigueNet, a novel multimodal framework for fatigue classification.
• To address limitations in current models by integrating diverse biosignals and complex signal interdependence.
• To develop an end-to-end system capable of real-time fatigue monitoring.

Main Methods:

• Utilized a combination of Graph Neural Network (GNN) and Transformer architecture.
• Extracted dynamic features from Electrocardiogram (ECG), Electrodermal Activity (EDA), Electromyography (EMG), and Eye-Blink signals.
• Employed adaptive feature adjustment and meta-learned gate distribution to capture temporal, spatial, and contextual relationships.

Main Results:

• FatigueNet demonstrated superior performance compared to existing benchmarks on the MePhy dataset.
• The model accurately detected fatigue levels across four distinct categories.
• Achieved an end-to-end latency of 50 ms per 20s window, proving real-time capability.

Conclusions:

• FatigueNet offers an improved approach to fatigue classification by effectively handling signal diversity and interdependence.
• The framework's real-time performance makes it suitable for practical fatigue monitoring applications.
• The multimodal approach surpasses traditional methods relying on manual feature engineering or single signal sources.