Fatigue Assessment from Facial Videos using Deep Neural Networks and Engineered Features Informed by Domain...

Area of Science:

• Biomedical Engineering
• Cognitive Science
• Machine Learning

Background:

• Fatigue significantly impairs cognitive and motor functions, increasing accident risks in critical professions like aviation and emergency services.
• Current fatigue assessment relies on subjective self-reporting, which is unreliable and prone to errors.
• There is a critical need for objective, rapid, and accurate methods to detect dangerous fatigue levels.

Purpose of the Study:

• To develop and evaluate a quantitative tool for rapid fatigue detection using facial video analysis.
• To compare the performance of a Long Short-Term Memory (LSTM) deep neural network against a Random Forest (RF) classifier for fatigue assessment.
• To establish a scalable and accessible method for fatigue monitoring in demanding occupations.

Main Methods:

• A quantitative fatigue vs. alertness evaluation tool was developed using less than two minutes of facial video captured via an iPad.
• Engineered features informed by domain knowledge were utilized.
• Classification performance was compared between a Long Short-Term Memory (LSTM) deep neural network and a Random Forest (RF) classifier.

Main Results:

• The Random Forest (RF) classifier demonstrated superior performance compared to the LSTM deep neural network.
• RF classifiers achieved average areas under the receiver operating characteristic curve of 0.72 ± 0.16 (11-fold CV) and 0.8 ± 0.12 (individualized 11-fold CV).
• Equal error rates for RF were 0.34 and 0.26, respectively, indicating robust classification accuracy.

Conclusions:

• The developed facial video analysis tool presents a promising, rapid, and objective approach for fatigue detection.
• The Random Forest model offers interpretability and outperforms LSTM in this preliminary study.
• Further data collection is planned to enhance generalizability across diverse populations.